# Shannon information and COVID-19

**URL:** <https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445>\
**Category:** Conversation\
**Tags:** Science, Article\
**Created:** [September 16, 2022, 10:14pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445 "2022-09-16T22:14:02Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![T\_aquaticus](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/t_aquaticus/32/1249_2.png) [@T\_aquaticus](https://discourse.peacefulscience.org/u/T_aquaticus)\
**Post date:** [September 19, 2022, 2:58pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/21 "2022-09-19T14:58:59Z")

</div>

Ted Schneider attempted to tie evolution and Shannon information together in a computer simulation.

> [@](#):
>
> How do genetic systems gain information by evolutionary processes? Answering this question precisely requires a robust, quantitative measure of information. Fortunately, 50 years ago Claude Shannon defined information as a decrease in the uncertainty of a receiver. For molecular systems, uncertainty is closely related to entropy and hence has clear connections to the Second Law of Thermodynamics. These aspects of information theory have allowed the development of a straightforward and practical method of measuring information in genetic control systems. Here this method is used to observe information gain in the binding sites for an artificial ‘protein’ in a computer simulation of evolution. The simulation begins with zero information and, as in naturally occurring genetic systems, the information measured in the fully evolved binding sites is close to that needed to locate the sites in the genome. The transition is rapid, demonstrating that information gain can occur by punctuated equilibrium.  
> [Evolution of biological information - PMC](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC102656/)

As @nwrickert states earlier, I think information in this context is an abstract concept that we humans have invented to understand the whole. Even in Schneider’s work we could easily argue that there are situations where a weakening in binding between a transcription factor and a DNA binding region would actually be beneficial. We could apply the same concept to SARS-CoV-2 immune evasion where a reduction in binding between specific antibodies and the S protein increases viral fitness. On the flip side, increases in binding between viral and host cell surface proteins could be beneficial.

What I don’t agree with is that there is a meaningful connection between the types of information modeled in these papers and the actual process of entropy. I do think physical information could be real (atomic spin, velocity, mass), but I don’t think this abstract type of information is concrete nor is it part of the 2LoT.

---

<div class="post-metadata">

**Author:** ![misterme987](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/misterme987/32/15207_2.png) [@misterme987](https://discourse.peacefulscience.org/u/misterme987)\
**Post date:** [September 19, 2022, 7:32pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/22 "2022-09-19T19:32:25Z")

</div>

> [@T\_aquaticus](#):
>
> Ted Schneider attempted to tie evolution and Shannon information together in a computer simulation.

Such a cool paper! Thanks.

---

<div class="post-metadata">

**Author:** ![RonSewell](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/ronsewell/32/6194_2.png) [@RonSewell](https://discourse.peacefulscience.org/u/RonSewell)\
**Post date:** [September 19, 2022, 7:32pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/23 "2022-09-19T19:32:45Z")

</div>

> [@Tim](#):
>
> Wouldn’t you need to create a universe where the 2LoT _doesn’t_ apply, to act as a control, and then see if life evolves in that universe?

Evolving is a process in time, and the 2LoT may be necessary for time or at least direction. So it may be asked, _who’s got time for that?_

---

<div class="post-metadata">

**Author:** ![Dan\_Eastwood](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/dan_eastwood/32/2368_2.png) [@Dan\_Eastwood](https://discourse.peacefulscience.org/u/Dan_Eastwood)\
**Post date:** [September 19, 2022, 7:50pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/24 "2022-09-19T19:50:00Z")

</div>

> [@T\_aquaticus](#):
>
> What I don’t agree with is that there is a meaningful connection between the types of information modeled in these papers and the actual process of entropy. I do think physical information could be real (atomic spin, velocity, mass), but I don’t think this abstract type of information is concrete nor is it part of the 2LoT.

Somewhere in my reading on IT I found mention of something which was roughly the equivalent of 2LoT for Information Theory. It applies (IIRC) to a change of information _relative to_ some starting point, not to an absolute increase or decrease as for physical entropy. If I can find it again I may have to get it tattooed on my arm for easy reference.

---

<div class="post-metadata">

**Author:** ![Dan\_Eastwood](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/dan_eastwood/32/2368_2.png) [@Dan\_Eastwood](https://discourse.peacefulscience.org/u/Dan_Eastwood)\
**Post date:** [September 19, 2022, 8:29pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/25 "2022-09-19T20:29:00Z")

</div>

NOT what I was looking for, but worth a read:

> **[Entropy and Information Theory: Uses and Misuses](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7514515/)**
>
> This article is about the profound misuses, misunderstanding, misinterpretations and misapplications of entropy, the Second Law of Thermodynamics and Information Theory. It is the story of the “Greatest Blunder Ever in the History of Science”. ...

---

<div class="post-metadata">

**Author:** ![Art](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/art/32/3805_2.png) [@Art](https://discourse.peacefulscience.org/u/Art)\
**Post date:** [September 19, 2022, 9:29pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/26 "2022-09-19T21:29:56Z")

</div>

> [@misterme987](#):
>
> I don’t know how, or if, it would be possible to test this hypothesis that life is a manifestation of the 2LT, but it’s certainly an interesting thought.

As I argued [here](https://discourse.peacefulscience.org/t/brian-miller-thermodynamics-and-the-origin-of-life/1616/2), to the extent that life is some sum of macromolecular associations and interactions, it is entirely correct to assert that life exists, not in spite of the SLoT, but because of it. (Read the whole thread, including the part before my first comment.)

---

<div class="post-metadata">

**Author:** ![Roy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/roy/32/3799_2.png) [@Roy](https://discourse.peacefulscience.org/u/Roy)\
**Post date:** [September 20, 2022, 1:22pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/27 "2022-09-20T13:22:57Z")

</div>

> [@Giltil](#):
>
> Shannon information measures the improbability associated with strings of characters but is unable to distinguish merely improbable sequences of characters from those that are meaningful or functional. For example, the two sequences below have the same probability, hence the same Shannon information, but only one conveys a meaningful message.
> 
> - in the beginning was the word
> - vtdke pushcgvt jifmzf gtlgd

“in the beginning was the word” doesn’t convey a meaningful message, so “vtdke pushcgvt jifmzf gtlgd” must be the one that does. What does it mean?

---

<div class="post-metadata">

**Author:** ![AndyWalsh](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/andywalsh/32/506_2.png) [@AndyWalsh](https://discourse.peacefulscience.org/u/AndyWalsh)\
**Post date:** [September 20, 2022, 1:27pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/28 "2022-09-20T13:27:07Z")

</div>

> [@Giltil](#):
>
> In fact, the more a genetic system has SIE, the more information it can convey. Conversely, the less it has SIE, the less information it can convey.

As being measured here, the greater the SIE, the more information that can be conveyed _per base_. With the small differences observed here, it would only take 12 additional nucleotides to get the total information back to the same level. And that’s well within the variation in genome length seen in SARS-CoV-2.

---

<div class="post-metadata">

**Author:** ![AndyWalsh](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/andywalsh/32/506_2.png) [@AndyWalsh](https://discourse.peacefulscience.org/u/AndyWalsh)\
**Post date:** [September 20, 2022, 1:31pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/29 "2022-09-20T13:31:54Z")

</div>

> [@Dan\_Eastwood](#):
>
> What do they mean by “carefully selected?” Why not a random sample? I’m very tempted to write a letter to that journal.

I wondered that too. So first I reproduced their results with the same 10 sequences.

 ![ReproduceOriginalFig](https://us1.discourse-cdn.com/flex016/uploads/peacefulscience/original/2X/e/e46bd9083e2608d4b7235ad1754c0506827d413e.png)

I didn’t scale by k\_b because it obviously won’t change the trend, and when I did so I got numbers on a very different scale than what the figure showed. However, the table shows the information entropy without the k\_b scaling, and my numbers were exactly the same as the table.

Once I had the data, I could also see that while the mutations increased over time, it was not a fully nested set of mutations. I got the impression the “careful selection” was intended to achieve that kind of nesting, but apparently not.

For there mutational bias hypothesis (which I also wondered about), here is the distribution of changes with respect to the reference sequence.

 ![MutDist](https://us1.discourse-cdn.com/flex016/uploads/peacefulscience/original/2X/e/e258ec15eadce46180a0dd8edffb1097d278724f.png)

Then I went to [NCBI](https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?VirusLineage_ss=Severe%20acute%20respiratory%20syndrome%20coronavirus%202%20(SARS-CoV-2),%20taxid:2697049&SeqType_s=Nucleotide) for more sequences. I used their filtering to restrict to complete genomes with no ambiguous characters. That still left a lot more data than I wanted to download, so I kept the 29,903 length restriction and only human samples from the oronasopharynx (no reason other than it was the only option that reduced the numbers further but left a useful sample). Could those choices introduce biases? Absolutely. I wouldn’t try to publish these results; I just want to quickly satisfy my curiosity.

Here are the same charts with this bigger sample:

 ![BiggerSampleOrigFig](https://us1.discourse-cdn.com/flex016/uploads/peacefulscience/original/2X/d/da09939643c38995dec32b426f4a21902c5f2f0e.png)

The overall trends are the same. I’d guess that the number of sequences thins out over time because some of the adaptive mutations are deletions, so that exact length of 29,903 becomes less common. The number of mutations in each sequence is with respect to the same reference sequence used in the paper. I could be underestimating by overlooking sequential mutations at the same location that might be discernible with a more sophisticated analysis than just computing the number of string mismatches.

Since we’re not dealing with a strict progression increase in mutations, a more relevant trend might be directly in number of mutations and information entropy, rather than indirectly comparing them by looking at their trends over time.

 ![BiggerSampleMutEntropy](https://us1.discourse-cdn.com/flex016/uploads/peacefulscience/original/2X/f/f725f3c36860a8f5f36b83b6629b5f25e7beacd8.png)

---

<div class="post-metadata">

**Author:** ![Roy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/roy/32/3799_2.png) [@Roy](https://discourse.peacefulscience.org/u/Roy)\
**Post date:** [September 20, 2022, 1:33pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/30 "2022-09-20T13:33:02Z")

</div>

> [@misterme987](#):
>
> Can anyone help me decode this paper?

‘We support ID and don’t care how bad our arguments are if we can get them published’.

---

<div class="post-metadata">

**Author:** ![Dan\_Eastwood](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/dan_eastwood/32/2368_2.png) [@Dan\_Eastwood](https://discourse.peacefulscience.org/u/Dan_Eastwood)\
**Post date:** [September 20, 2022, 1:47pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/31 "2022-09-20T13:47:32Z")

</div>

Wow @AndyWalsh, that is an impressive effort! (Here, have a badge 🙂 )

Cumulative mutations over time should be linear, but it’s good to confirm that.

> [@AndyWalsh](#):
>
> Once I had the data, I could also see that while the mutations increased over time, it was not a fully nested set of mutations. I got the impression the “careful selection” was intended to achieve that kind of nesting, but apparently not.

Great thought. It had not occurred to me to consider nesting.

> [@AndyWalsh](#):
>
> Since we’re not dealing with a strict progression increase in mutations, a more relevant trend might be directly in number of mutations and information entropy, rather than indirectly comparing them by looking at their trends over time.

A non-linear trend? I’m not sure what that means yet, will think on it.

---

<div class="post-metadata">

**Author:** ![Dan\_Eastwood](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/dan_eastwood/32/2368_2.png) [@Dan\_Eastwood](https://discourse.peacefulscience.org/u/Dan_Eastwood)\
**Post date:** [September 20, 2022, 1:50pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/32 "2022-09-20T13:50:39Z")

</div>

> [@Roy](#):
>
> ‘We support ID and don’t care how bad our arguments are if we can get them published’.

It’s probably just a bad paper.

**“never attribute to malice that which is adequately explained by stupidity.”**

[Hanlon’s Razor](https://en.wikipedia.org/wiki/Hanlon%27s_razor)

---

<div class="post-metadata">

**Author:** ![misterme987](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/misterme987/32/15207_2.png) [@misterme987](https://discourse.peacefulscience.org/u/misterme987)\
**Post date:** [September 20, 2022, 1:58pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/33 "2022-09-20T13:58:33Z")

</div>

> [@Roy](#):
>
> ‘We support ID and don’t care how bad our arguments are if we can get them published’.

Oh, no, I don’t think it’s an ID paper. Apparently it is pretty bad though. It’s too bad we don’t have the authors here to explain what their methodology was and why they drew the conclusions that they did.

---

<div class="post-metadata">

**Author:** ![Dan\_Eastwood](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/dan_eastwood/32/2368_2.png) [@Dan\_Eastwood](https://discourse.peacefulscience.org/u/Dan_Eastwood)\
**Post date:** [September 20, 2022, 2:05pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/34 "2022-09-20T14:05:44Z")

</div>

I have a suggestion for all. **Let’s put together a concise list of criticisms and questions, and contact the authors.**

A good starting point would be to summarize what’s already here.

---

<div class="post-metadata">

**Author:** ![T\_aquaticus](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/t_aquaticus/32/1249_2.png) [@T\_aquaticus](https://discourse.peacefulscience.org/u/T_aquaticus)\
**Post date:** [September 20, 2022, 2:36pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/35 "2022-09-20T14:36:28Z")

</div>

> [@Art](#):
>
> As I argued [here](https://discourse.peacefulscience.org/t/brian-miller-thermodynamics-and-the-origin-of-life/1616/2), to the extent that life is some sum of macromolecular associations and interactions, it is entirely correct to assert that life exists, not in spite of the SLoT, but because of it. (Read the whole thread, including the part before my first comment.)

Fully agree. Without entropy life couldn’t exist. It is the movement of energy from low entropy to high entropy that drives everything life does. Without a direction for energy to move along we couldn’t perform even the simplest metabolic tasks. For that matter, heat may not even reach us from the Sun.

---

<div class="post-metadata">

**Author:** ![misterme987](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/misterme987/32/15207_2.png) [@misterme987](https://discourse.peacefulscience.org/u/misterme987)\
**Post date:** [September 20, 2022, 2:36pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/36 "2022-09-20T14:36:43Z")

</div>

Yes, that sounds fair. It doesn’t seem right to keep bashing this paper without allowing the authors to explain first.

---

<div class="post-metadata">

**Author:** ![Giltil](https://avatars.discourse-cdn.com/v4/letter/g/5f8ce5/32.png) [@Giltil](https://discourse.peacefulscience.org/u/Giltil)\
**Post date:** [September 20, 2022, 3:30pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/37 "2022-09-20T15:30:54Z")

</div>

Thanks for your nice work. So it seems that your analyses confirm the authors conclusion, ie that the SIE of SARS2 RNA genome decreases with time, don’t they ?

---

<div class="post-metadata">

**Author:** ![nwrickert](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/nwrickert/32/810_2.png) [@nwrickert](https://discourse.peacefulscience.org/u/nwrickert)\
**Post date:** [September 20, 2022, 5:59pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/38 "2022-09-20T17:59:32Z")

</div>

> [@misterme987](#):
>
> It’s too bad we don’t have the authors here to explain what their methodology was and why they drew the conclusions that they did.

My wild guess is that they are probably engineers, and not biologists. They have perhaps been paying attention to news reports of mutations with COVID. But the news reports are really of those mutations which reached near fixation, so that could be what they were looking at.

---

<div class="post-metadata">

**Author:** ![Roy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/roy/32/3799_2.png) [@Roy](https://discourse.peacefulscience.org/u/Roy)\
**Post date:** [September 21, 2022, 3:24pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/39 "2022-09-21T15:24:33Z")

</div>

I’ve been doing some digging into the authors, and found this interesting [article](https://aip.scitation.org/doi/pdf/10.1063/5.0019941):

> Currently, we produce ∼10^21 digital bits of information annually on Earth. Assuming a 20% annual growth rate, we estimate that after ∼350 years from now, the number of bits produced will exceed the number of all atoms on Earth, ∼10^50. After ∼300 years, the power required to sustain this digital production will exceed 18.5 × 10^15 W, i.e., the total planetary power consumption today, and after ∼500 years from now, the digital content will account for more than half Earth’s mass…  
> …In conclusion, we established that the incredible growth of digital information production would reach a singularity point when  
> there are more digital bits created than atoms on the planet.

Paging Malthus…

---

<div class="post-metadata">

**Author:** ![Dan\_Eastwood](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.peacefulscience.org/dan_eastwood/32/2368_2.png) [@Dan\_Eastwood](https://discourse.peacefulscience.org/u/Dan_Eastwood)\
**Post date:** [September 21, 2022, 3:41pm UTC](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445/40 "2022-09-21T15:41:50Z")

</div>

> [@Roy](#):
>
> > …In conclusion, we established that the incredible growth of digital information production would reach a singularity point when  
> > there are more digital bits created than atoms on the planet.
> 
> Paging Malthus…

I always save the contents of my [**bit bucket**](https://en.wikipedia.org/wiki/Bit_bucket) for recycling (CoSci joke).

My first CoSci instructor told many tales, including one about keeping an actual bucket in the computer cabinet so the seniors operators could tell the new guys to “empty the bit bucket.” 🙂

Paging @AllenWitmerMiller

[Previous page](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445.md?page=1)

[Next page](https://discourse.peacefulscience.org/t/shannon-information-and-covid-19/15445.md?page=3)