I’m a bit confused because people have been talking about MI without referring to MI between which sets of data. In this post, whenever I switch to LaTeX, I’m going to use the strict language of set theory, without any reference to MI, so as to keep any obfuscating language away.
Recap of my personal understanding
As I understand it, Josh has argued that even if (C1 \cap G), (C2 \cap G) , and (C1 \cap C2) all decrease (meaning: the total area of the circles which are only overlapped by at most one color increases), (C1 \cap C2) \setminus G (which I take is what Eric calls CMI) may increase. I believe Eric agrees with this basic mathematical statement.
As Josh explained, why is it possible that (C1 \cap C2) \setminus G increases? Intuitively, what is happening is that somehow the two cancers mutate in a way that is somewhat correlated with each other. From some biological reasoning, this is explained as a result of driver mutations that are essential for the cancer to function. Because of this, Josh names (C1 \cap C2) \setminus G as functional information or FI. Eric agrees that it is possible for random + deterministic processes to result in an increase of FI (=(C1 \cap C2) \setminus G ).
Up to this point, I think this is fairly clear. Please correct me if I’m getting anything wrong.
Eric’s Objections
I’m not sure I fully understand these objections. Here is my interpretation.
Using my notation, the above equation is expressed as:
(C1 \cap C2) \setminus G = (C1 \cap C2) - (C1 \cap C2 \cap G)
(This is trivially true, of course, if you check Josh’s image. It also follows from basic set theory identities. So we all agree on this.)
The only way I understand Eric’s objection to this is that somehow, (C1 \cap C2) (which I understand is what he calls “absolute MI”) is limited by information non-growth and cannot increase.
But even if this is true, the above equation doesn’t imply that (C1 \cap C2) \setminus G cannot grow, even as it is bounded above by (C1\cap C2). Even if (C1 \cap C2) stays the same, or even decreases, it is still possible for (C1 \cap C2) \setminus G to increase. How so?
With every iteration, even as the first term (C1 \cap C2) continues to decrease, the second term (C1 \cap C2 \cap G) can decrease faster than the first. In other words, the two cancers are both moving away from the germline much faster than they are moving away from each other. Hence, the difference between the two terms can increase - in other words an increase in (C1 \cap C2) \setminus G, or functional information.
To picture this in your head, imagine the blue and green circles moving further away from each other, but also jumping together downwards, far from the germline. Although the horizontal width of the overlap between them has decreased, the vertical height has dramatically increased, leading to an overall increase in area!
This is, as I understand it and as @Dan_Eastwood pointed out, akin to the realization that the 2nd law of thermodynamics doesn’t disprove evolution, because local entropy may decrease. Of course, there is an limit to how much it can decrease. The analogy would be that if the green and blue circles have completely separated away from the red one, then from that point onwards, FI cannot continue to increase. This is the “heat death of the universe” scenario.
The only way to mathematically object to this is to show that it is not possible for the second term to decrease faster than the first. But as I understand it, this is a scientific, not mathematical question.
I would like to invite Eric and Josh to scrutinize and correct my attempt at understanding the dialogue here.