Why Axe’s response to “objection 3” doesn’t work.
What is objection 3? Axe writes:
Objection 3: Because Axe measured mutational sensitivity from a weakly functional starting sequence rather than the fully functional natural enzyme, the mutants he generated were inappropriately disadvantaged, and this is why he arrived at such a low value for the prevalence of functional sequences.
According to Hunt, I “molded a variant that would be exquisitely sensitive to mutation.” [2] Venema expressed the same concern, that the starting sequence I used was “intentionally ‘hamstrung’ with multiple mutations to render it far less functional than its natural counterpart.” [7]
Both Hunt and Venema seem to think the outcome would have been more favorable (i.e., functional sequences would have been more prevalent) had I used the highly proficient natural enzyme as a starting point rather than the handicapped version. Actually, as a demonstration will show, the opposite is true.
In Axe’s response to objection 3 he intends to show that, had he used the wt enzyme in his experiment, he would have obtained an invalid result.
I will show that Axe’s analogy that claims the result would be invalid simply assumes that as his conclusion. His analogy doesn’t show that the result with the actual proteins would have been invalid, it assumes it. And further, if Axe’s analogy-argument here was correct, it would undermine his own experiment too, giving us zero reason to think ANY number he has estimated can be trusted.
In Axe’s analogy to his lab experiment, he breaks a 42 character sentence into six fragments of seven characters each, then mutates each fragment individually and puts it back into the full sentence context, and then assessess whether the sentence has retained readability with this specific 7-character fragment having mutated.
He does this for each of the six fragments. Importantly, the 35-character remainder of the sentence is unaltered in his analogy. He does this to see how many mutations each fragment can tolerate before the full 42-character sentence loses readability. He doesn’t really show where that limit is, but suggests it is somewhere in the range of 2 to 5 mutations for each 7-character fragment.
When he puts all four mutated fragments together, it’s gibberish Axe writes:
Here’s the problem, though. In calculating this higher P value, we have effectively assumed that the typo rate we found to be tolerable in the individual collections (two typos per seven positions) remains tolerable when we apply it to the full 42-character string. However, when we try this by generating full-length strings with twelve typos (maintaining the 2-in-7 ratio), we get completely unreadable gibberish:
So Axe says because the result is gibberish, it shows we can’t extrapolate the per-fragment tolerance to the entire sentence. Doing this, the result is invalid. The mutations in each fragment were tolerable only because the remainder of the unaltered sentence retained enough meaning to sort of “buffer” against the loss of meaning each mutation would create in an individual fragment:
Evidently we have made a mistake, because mutants that ought to be readable according to our calculation clearly aren’t readable. As you may have guessed, the mistake is that the 2-in-7 typo rate was tolerable only because it was restricted to a narrow section of seven positions. The fact that the remaining 35 positions were without error compensated in large measure for the errors in the mutated section.
So Axe’s analogy here actively relies on our abilities to assess the readability of a sentence. We are supposed to think that because we can see that introducing the same number of mutations (say 2 in 7) that were tolerated in each 7-character sentence fragment—when introduced at the same ratio (2 in 7) into the full sentence—results in loss of readability of the sentence, then we should now think that we would have obtained the same invalid result with the protein enzyme.
That the enzyme’s “readability”, which would have been it’s enzymatic function, would have been lost too if we had put the same number of mutations found to be tolerated for individual enzyme fragments in the context of the wild-type enzyme, back into the full enzyme sequence at a similar ratio (2 mutations in every 7 residues, say).
This analogy therefore assumes that sentence readability and character typos must function analogously to how mutations affect protein enzymatic function. First of all Axe gives us zero reason to think that is the case anywhere. That means his analogy, as it currently stands, is flat out question begging his conclusion. He has not shown the result for a protein enzyme using a similar method would be invalid. He has merely declared it. His analogy is not grounded in enzyme biophysics.
Axe gives us no reason to think that the changes to the readability of a sentence borne from typos has enough of a meaningfully comparable relationship to the functional effects of substitution-mutations on enzyme functions, such that his analogy demonstrates what results we would obtain had we performed a similar experiment with actual enzymes.
To justify his conclusion, Axe would need to show that something like the following would be true.: That the wild-type enzyme is already so close to the edge of functional collapse that its apparent mutational tolerance is mostly an artifact of global buffering, and therefore fragment-level tolerance vastly overestimates true functional density. But he never demonstrates this either experimentally or theoretically. Rather, he merely asserts it using the sentence analogy.
Just one argument to think there are some dis-analogous affects in play: In real protein sequences specific residues can not only compensate for the deleterious effects of mutations at other locations, they can even some times combine in a way where two individually deleterious mutations become beneficial when together in the same protein’s amino acid sequence. But misspelings in later words generally never compensate for misspellings in earlier words, much less do they ever combine to improve sentence readability.
This is not to say that there is no analogous relationship at play (as we randomly increase the number of mutations, we would also expect protein function to eventually degrade), but Axe’s analogy does no work to establish that it is even close to the relationship between mutations and enzyme function. It is not at all clear from his analogy that a more stable enzyme being tested against a ~10ug/ml concentration of antibiotic would produce skewed results.
Having now dealt with how Axe’s analogy simply begs the question, let’s move on to see how, if his argument worked, it would undermine his own experimental results. In that he would be forced to conclude his own experiment has produced a skewed result too.
This one should be fairly obvious. If we can’t extract a valid result by extrapolating the mutational tolerance of a single fragment, to the entire sequence, then that would apply both to the temp-sensitive enzyme and to the wild-type.
Either way, he has a sequence that is more tolerant to mutation to begin with before mutating the fragments of it, then mutates fragments of it even further. At best Axe could argue that the magnitude of error would be larger if he had used the wt enzyme as a context for the fragment, since this could presumably buffer more mutations before losing function, than his temp-sensitive enzyme.
But since his temp-sensitive enzyme also has some, though lower, capacity to buffer mutations (if it didn’t he should find a pass rate of 0%, not 38%), his own analogy with sentence-readability should still apply and he must now argue that he is deriving an invalid result even with a temp-sensitive enzyme. What reason do we have to think that an enzyme being tested at 25 degrees C against a concentration of 10 ug/ml, that yields a 38% pass rate, is closer to the correct value for functional and nonfunctional sequences, than if we had used the wt enzyme against the same or a higher concentration? His analogy doesn’t tell us, rather if we believe it despite being given no reason to, it simply undermines that claim.