Thanks to everyone who contributed to this discussion. Given its length, my reply will have to be divided into a few parts. What follows is part 1.
Preliminary note to T. aq. Your question about statistical methods raises important but complicated issues that require a separate thread, which I hope to begin in the next couple of days.
So who was the mystery philosopher of evolutionary theory, what did he say about the Precambrian rabbit test, and why does it matter anyway?
Elliott Sober of the University of Wisconsin. Here’s what he (and his co-author Mehmet Elgin) said, in context:
Evolutionary biologist John Maynard Smith (1986, 5) says that “the theory of evolution is not falsifiable in the sense required by Popper.” He says that the theory is a “logical deduction” and that Popper is demanding something more—that “a scientific theory must say something about the world, and not merely about logical necessity.” Maynard Smith goes on to say that “Darwinism as a testable scientific theory can take various forms. I will give it first in the form in which Darwin himself proposed it, and then in a ‘neo-Darwinist’ form in which most biologists hold it today.” Maynard Smith then says that the hypothesis of common ancestry is “clearly falsifiable . . . as a single fossil rabbit in Cambrian rocks would be sufficient.” This is a double mistake. The discovery that Maynard Smith describes would require an important reworking of when various taxa first appeared and how different taxa are genealogically related, but neither of these changes shows that the hypothesis of common ancestry is false. (emphasis added)
Complete text available here: Elliott Sober - Selected Papers
One could go many directions from a point like this. Maybe Popperian falsificationism is the wrong philosophy of science to adopt (as Sean Carroll has been saying lately), or maybe Maynard Smith, like the many other evolutionary biologists who have cited the Precambrian mammal test, was just confused (or maybe both). I want to go in a different direction, however. Popper and Maynard Smith aren’t my main interest.
Uncertainty about how to test universal common descent (UCD) persists 159 years after Darwin introduced the hypothesis. Ask a garden-variety evolutionary biologist about how he would know if UCD were false, and odds are, he’ll give some variant of the Precambrian rabbit test. But Sober thinks the test is a mistake, as do many of the people contributing to this thread. Why?
Until recently (see parts 2 and 3 of my reply), almost any biologist of repute would have said that UCD is a fact, and we don’t test facts; we take them as our unquestioned starting points in building theories. Within the ID research community, there’s significant debate about this, because some ID advocates, such as Mike Behe and Michael Denton, accept UCD, whereas others (e.g., Jonathan Wells, Doug Axe) are skeptics; others, such as Ann Gauger, can make a case on either side and remain undecided.
For several years, Discovery Institute has run a summer natural sciences seminar for students from around the world, to discuss the main issues of ID. My teaching responsibilities in the seminar include evo-devo and UCD (for and against). I found that, every July, a sizable percentage of my students accepted UCD, or found its factual status as plausible as any possible challenge to that status.
Here’s the sort of dialogue I’ve had with these students:
Student: We know that UCD is true because the genetic code is universal.
Me: The genetic code is not universal (https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi).
Student: These are minor exceptions. The genetic code is near-universal.
Me: “Near-universal” is a nonsense phrase, like “near-pregnant,” or "the number six is near-odd – on a good day it’s almost seven."
Student: You know what I mean. The variant codes are not different enough from the universal code to qualify as inconsistent with UCD.
Me: So what variant code would be different enough to qualify?
Student: A code that could not have evolved from the universal code present in LUCA.
Me: Your answer entails that you know that the variant codes in the NCBI collection evolved from the universal code, and thus, how they evolved.
Student: I don’t know how those codes evolved, personally, but I’m sure someone does, because the biological community accepts UCD as fact.
Me: We can discuss the proposed hypotheses of code evolution later. They all have problems. But isn’t the real issue that UCD doesn’t predict variant codes? Hence, the empirical confirmation of UCD by the universality of the code, 1966 to 1985, was illusory, because UCD is equally well confirmed by the non-universality of the code, 1985 to 2018.
Student: UCD is not a predictive theory, at least not in the way you want. It’s an explanatory framework. And UCD doesn’t need confirmation any more, because everything we know in biology supports it.
Me: (sigh – thinking, what the hell is an explanatory framework?)
I realized that, to brings students face-to-face with the shortcomings of this "explanatory framework but not predictive" take on UCD, I would need to show them just how little UCD actually told them about the biological world, if they didn’t already know the answer from some other source.
So I devised a handout entitled "Phylogenetic Exercises," based on a simple device.
Take any biological character thought to be shared by two or more species, or any published phylogeny, and mask a portion of the character or character state distributions at the terminal taxa. Then ask someone to predict what characters or character states they should find on those masked branches, using only the phylogeny in question (i.e., no cheating by looking at PubMed or googling the answer).
Josh Swamidass has tried his hand at these exercises. I’ll send them to anyone who is interested (contact me at nelsonpa@alumni.uchicago.edu).
While I need to re-design the phylogenetic exercises for a much larger sample, and need the help of a good statistician with other aspects of the experimental design, from casual inspection of the results so far, the answers look to be no better than random.
In other words, if one does not know the outcome already, UCD – whether in its fully universal form, from LUCA, or for some clade within the Tree of Life, such as the vertebrates – gives no guidance. One may as well toss a coin or roll a die.
Turns out this result has been known for a long time:
A given tree hypothesis is logically congruent with any specific observable evidence of character state distribution. In other words, a given tree, in combination with decent with modification as background knowledge, does not prohibit any specific character state distribution pattern (Farris, 1983; Sober, 1983). As there is no deductive link between any tree hypothesis and any specific character state distribution there exists no direct empirical test of hypotheses of monophyly (i.e., clades) sensu Popper (Sober, 1988; Rieppel, 2003)—one cannot think of any observation, which, in case it would represent a true statement, would allow to conclude the falsity of a clade or a given cladogram through modus tollens. (emphasis added)
From here:
https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1096-0031.2007.00169.x
Here endeth part 1. Thanks again to everyone who commented. More to follow, gotta do my day job or Discovery Institute will come looking for me with a Doberman.