Side Comments on Mung's Information Primer

You constantly reprimand people for their math errors, yet make very confident erroneous statements like the above about basic information theory concepts. Perhaps a change in tone is warranted.

Mutual information:

I(X;Y) = \sum_{x\in X}\sum_{y\in Y} p(x,y) \log\left(\frac{p(x,y)}{p(x)p(y)}\right)

Joint entropy:

H(X,Y) = \sum_{x\in X}\sum_{y\in Y} p(x,y) \log(p(x,y))

Not the same, and for an important reason. Mutual information is the Kullback-Liebler divergence of the actual joint distribution of X and Y from the joint distribution of X and Y if they were independent.

Good point. That was an error. Spoke to soon. See how easy it is to correct me when I make a mistake?

1 Like

So a change in tone, then?

@EricMH there are several threads devoted to your questions. This one is for @Mung. You should not post here. Any tone you detect does not involved you.

I encourage you to be as quick at retracting errors I was just with you. Everyone makes errors. The people you should trust are those who are easily correctable and quick to make changes. Right now, there appear to be several errors that you are caught in that are taking months to resolve.

1 Like

@Mung those are good comments, I will edit when I am back to a real keyboard. I avoided “entropy” because I’m less familiar with the thermodynamic interpretations, but I need to add that in somehow.

@swamidass thanks, in my effort to use simple tetms that one backfired. Suggestion?

Rather than doing the exercise for @mung, help direct him on how to do it for himself. Don’t do his homework for him. Help him figure it out. This isn’t rocket science. Mistakes will be made, but he has to work it out if he wants to learn. I think he is actually quite capable of this.

1 Like

@EricMH, I made a side comment thread for you post in. Please do not post in the main thread any more. Please respect @Mung’s effort to figure this out for himself.

You have certainly not been quick to retract errors. Off the top of my head:

  1. You can empirically disprove a mathematical proof
  2. ASC over estimates CSI
  3. A halting oracle is logically impossible
  4. You can disprove ID

How does this relate to your last sentence?

You have eventually stopped stating these errors, but only after I constantly held you accountable. And you have never publicly admitted you are wrong on those points, although at least on point #3 you eventually offered your own argument that a halting oracle is possible.

@swamidass You also state that there are multiple errors I am not admitting. What are they? I can address them if you will tell me.

Humans are imperfect halting Oracles and can thus increase total information?

That is speculation, but it is not outright error since an imperfect halting oracle is logically possible.

An “imperfect halting Oracle” isn’t even an oxymoron. The whole point about a halting oracle is that it can reliable determine whether any process can halt. An ‘imperfect halting oracle’ could logically be anything that isn’t a halting oracle. My dog is one such example.

It’s ‘all or nothing’ by definition and in application.


Yes, and an imperfect halting oracle is an undecidable subset of those decisions.

This would be comical retelling if wasn’t tragic. None of these things are errors.

This is clearly true, and not an error.

We came to settle together that this was true. I can retrieve quotes if you doubt it.

That is entirely true. I even presented the proof to you.

I’ve pointed out that I can disprove specific ID argument, and I can. This is true.

Disagreements are not errors @EricMH. In fact, you have vacillated back and forth on three of these things, admitting (correctly) you had made an error after quite a bit of conversation, only to forget it now.

@EricMH there is a large picture that seems to be lost sight of here.

The only reason halting oracles are important is because:

  1. Perfect halting oracles (i.e. halting oracles) are required for one proof how information (in one narrow sense) cannot be created.
  2. We can prove that halting oracles are not any combination of determinism + randomness because they are a logical contradiction.

You rely on those two claims to prove that any “information” is a signature of intelligence. That is the only reason we were discussing it.

So now, you say this, ignoring the actual definition of a halting oracle and the proof that it is a logical impossibility.

Which is why @Argon counters:

To be clear, there is no proof whatsoever against determinism + chance making an “imperfect halting oracle,” to use your neologism. If you grant that “imperfect halting oracle” can produce information, as you have, the whole argument falls apart. I’ve explained exactly where your error is. You are using “information” in two contradictory sense: the theory of information that arises from an omniscient total view of a system, vs. the empirical reality of measuring part of a system with imperfect knowledge.

That is the error in your thinking. What you think is a “rescue”, that there are “imperfect” halting oracles, undermines the second point, that halting oracles don’t exist in nature. Therefore the whole argument false apart. This is an example of us mathematically proving that an ID argument is in error. It leads to direct contradiction.


Alright, I take it back. I thought you had retracted this error. At least you seemed to do so in private communication.

You are quite wrong on that point. Allow me to now act aghast you would believe such a thing as a tenured professor and expert in information theory and say this throws your whole credibility into doubt, as you’ve done so often against me.

Sure, quote ahead. The math is clear. I guess it’s because you don’t think math holds in reality :smile:

Whoa, crazy man! A complete about face from the thread where you claimed God could be a halting oracle. Oh well, I guess that’s what happens when you believe math doesn’t apply to the empirical world.

Anyways, you are also clearly wrong on this point. Halting oracles are not impossible. Your “proof” is just a circular argument that halting oracles are impossible because everything is a Turing machine.

Ah, that is a bit of goalpost shifting there. We started our interactions on this forum over the claim that you could disprove the central information argument of ID, which you have never done in the many weeks we’ve gone back and forth here. Clearly you can do no such thing. But again, not surprising you’d make contradictory claims if you think math doesn’t apply to the empirical world.

As you insisted that I supply specifics when making such accusations, you should do the same here. Unsubstantiated allegations are not a trustworthy thing to do. But, don’t bother, because as I said I’m leaving this quite passive aggressive forum.

Good math applies to the empirical world. The stuff you’ve presented so far, no so much. Like your claim you have mathematically proven real world evolutionary processes can’t add information to a genome. That was a real beauty. :slightly_smiling_face:

1 Like

The statement is true, but my argument was flawed because I neglected the important qualification that the target must be fixed.

It’s already published. Check out Leonid Levin’s paper: Think about how it applies to evolutionary processes.

I haven’t seen anything you or anyone has published in the primary scientific literature proving real world evolutionary processes can’t add information to a genome. I’m sure you can “prove” all sorts of things in your extremely limited hypothetical math world but applying it to the real world is another thing entirely. You have come nowhere close to doing so.

That’s not an imperfect halting oracle. Perhaps you missed my use of the term “undecidable” or don’t know what it means. I don’t know. It’s clear you are not interested in understanding what I have to say, but are merely looking for opportunities to ID bash, or “ruthlessly destroy bad ID arguments” as you like to call it. If you thought about it for a second you’d see an “imperfect halting oracle” is a perfectly coherent concept, i.e. just take the infinite set of decision problems for a perfect halting oracle, and remove one element. Still an infinite undecidable set, but no longer coextensive with that of the perfect halting oracle. See? Pretty easy.