The evidence suggests that Neanderthals were not human. There are several categories of evidence for this, but I’d like to focus on the very strong genetic evidence which points to this conclusion after just one point from morphology.
Of course humans have been changing somewhat for a while. Our modern globular brain shape did not even work its way into our phenotype until after 36,000 years ago, a date which some would say was near or after the date Neanderthals went extinct. I am not even convinced that the physical brain is what truly makes us human. But to the extent that it is brain shape may be even more important than brain size. Hominid brain size did not change much in almost half a million years while what humanity has accomplished from the time modern brain shape occurred to the present is orders of magnitude greater than those of other hominid groups given that same amount of time with a similar brain capacity.
Scientists tell us that all living Eurasians have an average of 1.5% of their genes from Neanderthals due to admixture events which occurred approximately 55,000 years ago. My own genome confirmed that this is the percentage in me. There is a caveat to this which I would like to address in a bit, but let’s go with that number for now. The discovery of a 40,000 year old Romanian male with somewhat more recent Neanderthal introgression seems to represent a genetic dead end who left no living descendants (else we’d find people with longer such gene segments than we do).
Indeed a comprehensive study showed that the lack of Neanderthal mtDNA (inherited from mothers) in modern Europeans despite thousands of years of the two living side by side indicated possible sterility problems between human males and neanderthal females and that the maximum possible number of such hybridization events over 12,000 years was 120.
Other studies indicate fertility issues going the other way. The human Y-chromosome is lacking in the “Neaderthal” genes present in the rest of the genome. The researchers concluded that modern human females and neanderthal males were not fully compatible. The most parsimonious explanation was that such unions could not produced fertile male offspring.
Put the two studies together and the lack of Neanderthal genes in both mtDNA and Y-chromosome DNA is consistent with the idea that the only pairing that really “worked” was a male neanderthal mating with a human female and then only for producing female offspring.
I put the word “worked” in quotes because there is other evidence to show that it did not work well. This study concludes “Much of this Neanderthal DNA appears to be deleterious in humans, and natural selection is acting to remove it.” The paper suggests that this was a consequence of their smaller group size- allowing more deleterious mutations to stay in the genome rather than a fundamental weakness in genes making the species jump. It bases this conclusion solely on their finding that these mutations were only weakly deleterious. That does not add up to me because the other studies showed that plenty of other mutations seemed to be strongly deleterious and thus are no longer around to be counted!
But even if the researchers got that right, it is still a consequence of humans having a sense of connectedness and sociability in a way that other hominids may have lacked. This helped us live in larger groups which helped keep weakly deleterious genes from fixing. What we know is that while these genes “worked” in producing fertile offspring in some cases, much of the load was at least mildly deleterious.
Now I say that the genes are being weeded out slowly over time, but it now appears that some of those genes which the OOA humans got from (presumably) Neanderthals was actually the ancestral condition of the gene in both species which had been lost in modern humans. IOW, the Neanderthals did not give them neanderthal genes, some of the 1.5% is simply the genes our species once had in common with them but lost being returned to us through these events. Either that or they were ours all along and returned to us through an earlier hybridization event. So even a large proportion of the “Neanderthal” genes we wind up keeping may not really have been exclusively “theirs” anyway. Thus whatever the true figure of percentage of neaderthal genome possessed by the average Eurasian, it is liable to be lower than the oft-cited 1.5%
So the scope of all the evidence indicates that successful human-neanderthal hybridization was a limited event which ended before humans obtained their present brain shape and the explosion of human culture around 36,000 years ago in the Upper Paleolithic Revolution. IOW even if what we call humans 55,000 years ago had instances where the barriers to fertility were overcome and fertile offspring were the result there is no reason to think that living humans and neanderthals, if they could be brought back, could do so. The fact that the researches in the study I cited earlier said that they could not detect any evidence of such interbreeding in living Europeans from the time humans and neanderthals lived side by side in Europe for thousands of years shows that there was no repeat of what probably happened in the mid-east 55K ago.
If any group can think outside the box, even our own boxes, its this one. Something changed in humanity in the Upper Paleolithic, and that even changed the way they interacted with other homininds. Lions and tigers have about the same fertility issues that the research indicates that archaic homo Sapiens and h. Neanderthal had, and we all know they are not the same species.