@Patrick, I wouldn’t be looking for Shannon information in the box, either. Or the switch. 
1 Like
OK, but to be clear, I think that specifying what the probabilities are (eg ((X=‘tails)’ = 0.5 or instead of listing specifying how to calculate them given the scientific mechanisms is in the “about”.
For Shannon math alone, we just have the an uninterpreted probability distribution as part of the definition.
Maybe that is what you meant by ‘about’ too. But in that case you left out the sample space interpretation, eg sample space is {‘heads’, ‘tails’} and prob dist is P(X=‘heads’)=0.6, P(X=‘tails’) = 0.4.
2 Likes
We need to be careful here not to confuse meaning with information, and how information is coded. For Shannon Information the only concern is how many bits are needed to transmit that code.
SI always deals with transmitting numbers. We might assign some additional meaning to a particular number, but that meaning is irrelevant to SI.
Now imagine that Bruce and Dan are exchanging coded messages, and Mung’s job is to the carry those messages back and forth between us. Mung doesn’t understand the codes, it’s just his job to write the messages down and carry them back and forth, delivering them accurately.
Mung, being an observant fellow, notices that many of the codes we use are repetitive, and that he can use a shorthand method to RECODE our messages into a shorter version, then reverses his shorthand to deliver an accurate massage. Mung has invented data compression, and his job becomes much easier by doing this.
In the example above, Bruce and Dan might be exchanging long but repetitive messages, for example: the Gettysburg Address in an ASCii text file written out binary. Mung doesn’t the understand the message, he just sees zeros and ones, but he quickly discovers the repetitiveness, so every time he sees this same long sequence of symbols he jots down “011”. On delivery reads back the shorthand to deliver the original message. The rules of Mung’s shorthand, written down, are a coding scheme. Written as a computer program, the length of this program describes the Kolmogorov Information (or Algorithmic Information) of the population of messages (a probability distribution) that Bruce and Dan exchange.
KI removes the requirement that message can only be numbers, and allows them to have any meaning that Bruce and Dan agree to in advance.
Dan and Bruce can exchange meaningful message much more efficiently with Mung’s help. Mung doesn’t need to know the meaning of the messages to do this, he just needs to observe the probability distribution of messages and invent a clever way to recode them.
2 Likes
But I get paid based on how long the message is. Why on earth would I find a way to make it shorter?
2 Likes
You should run for congress! 
2 Likes
Cracking open some books that I expect provide evidence for common ancestry and lo and behold the first one seems quite enamored with information in the genome. Doesn’t define information (yet) but says there are different types of information in the genome.
Also appears to have not received the message that Darwinism is dead.