On another thread, @nwrickert raises some interesting distinctions:
He is not defining information. He is defining a quantity which measures how much information. And that’s what I have always understood to be Shannon’s entropy.
how much information = entropy – Yes to that.
information = entropy – No to that.
Entropy can’t be rate. The rate would be in something like “bits per second”. But Shannon entropy is supposed to be a dimensionless ratio, so the “per second” part doesn’t fit.
@nwrickert, if you’d like, we can hash this out here. Entropy and information have a scale determined by the base of the logarithm used. So they do have dimension. With log base 2, they are in units of bits. With the natural log, they are often called “dimensionless”, but the fact is that information in bits can be trivially rescaled to be “dimensionless” precisely the same way as entropy.
The equations for measuring entropy and for measuring information are identical. So we can talk about the rate of entropy across a channel in the same way as we talk about the rate of information as a channel. They are exactly the same things, with different semantic gloss.
In context, Shannon is just defining the total information transmitted in a period of time, which he divides by the number of seconds. The exact same thing can be done with entropy to get a rate. Information content = entropy amount.
What am I missing about what you are saying?