The fact that you have not directly observed something to occur is not a reason to think it can’t happen. Straightforward non-sequitur. We need know nothing else to see your position stands on thin air.
Second, those aren’t lacking. Changes in chromosome numbers, loss of genes through both wholesale deletion and pseudogenization (or gain by duplication), and these having no obviously deleterious phenotypic effect, are all observed realities.
So that’s a no, you have no such reason.
Edit: See for yourself: Evolution by gene loss | Nature Reviews Genetics
Gene loss and dispensability
The pervasiveness of gene loss throughout evolution leads to the fundamental question of how many genes can readily be lost in a given genome. Intuitively, the answer to this question depends on how many genes are actually essential for a given organism, and therefore cannot be lost, and how many genes are to some degree dispensable, and therefore susceptible to being lost because their loss has no impact or only a slightly negative impact on fitness, at least under certain circumstances (FIG. 2).The knockout paradox. Gene dispensability is a meas-ure that is inversely related to the overall importance of a gene (that is, gene essentiality), and this measure has been approximated by the fitness of the corresponding gene knockout strain under laboratory conditions38,39. Understanding which genes are dispensable or essential by linking genotypes with phenotypes is one of the most challenging tasks in the field of genetics and bio-medicine in the twenty-first-century post-genomic era. This understanding is important both theoretically, such as when defining the minimal genome for a free living organism40, and practically, such as when identifying all essential genes that are responsible for human diseases41. Historically, Susumu Ohno not only pioneered the idea that gene duplication was an important evolutionary force, but in 1985 he also pondered the concept of gene dispensability and suggested that “the notion that all the still functioning genes in the genome ought to be indispensable for the well-being of the host should be abandoned” (REF. 42). The emergence of large-scale gene targeting approaches has facilitated the calculation of the number of genes that are globally dispensable in a given genome in certain conditions. Thus, systematic large-scale approaches that involve single-gene deletions in Escherichia coli and other bacterial species showed that only a few hundreds of genes are essential, suggesting that nearly 90% of bacterial genes are dispensable when cells are grown either in rich or minimal mediums39,43,44. The high degree of global gene dispensability found in bacteria is consistent with findings from systematic gene deletion screens in Saccharomyces cerevisiae and Schizosaccharomyces pombe. These screens revealed that approximately 80% of protein-coding genes are dispensable under laboratory conditions45,46. Following the same trend, large-scale RNA interference approaches in C. elegans47,48 and D. melanogaster 49 suggested that 65% to 85% of genes, respectively, are dispensable in these organisms, and similar figures were obtained in mice by the Sanger Institute Mouse Genetics Project50. Recent attempts to test for gene essentiality in humans using gene trap and large-scale CRISPR–Cas9 screens suggest that approximately 90% of tested genes are dispensable for cell proliferation and survival, at least in human cancer cell lines51–53. These surprisingly high values of seemingly dispensable genes in different organisms and their tolerance to inactivation have been referred as the ‘gene knockout paradox’ (REF. 38). Two main factors have been provided that may account for this observed gene dispensability54: mutational robustness and environment-dependent conditional dispensability.