We Are Mystified by Eric Holloway

[swamidass (S. Joshua Swamidass)]

@swamidass

So it is both a false claim, and if it were true, that will still demonstrate the error in his implementation.

Somewhat stunningly to me too is that I explained already how to fix it for some narrow cases.

@swamidass

This is just an error with LZ compression because it will always be close to zero, except in some boundary cases. LZ can’t pick up on long range repetitions, so it will just fail to compute MI. A revised version that might work in some limited cases is:

I_LZ(X) - I_LZ(X ^ Y)

Here the “^” is the XOR on the bit strings, returning a bit string that is 1 every position the two strings are different. This function would actually be able to compute the mutual information, assuming the two are perfectly aligned. So, for example,

if X and Y are equal to: TAGGCTTAGGCAA

Then X^Y, would be a very easy to compress string.

0000000000000

This is good, making it so that MI(X,Y) is equal to I(X) and I(Y). It is very easy to break this however.

This actually would produce a reasonable MI in @EricMH’s simulation , though, as I said, is easy to break with a different E function (a rotation, and many other sorts of shuffling). What just is hard to for me get my head around is @EricMH’s reasoning here. It really does look like epistemic closure, but there is such confidence exuding from him I’m unclear exactly what is going on in your head, @EricMH. I hope you can enlighten us.

There is a point here too. It is not possible to fix for all cases, just some. You have to actually model the process to know the compression size / probability function.