Glancing through a couple other parallel discussions in this forum (‘IDist Disbelieves …’ and ‘Do I Fudge My Math?’) I am reminded of the merit of very carefully and even pedantically, laying out the grounds for the approach I take to define, measure, and estimate functional information, in an effort to avoid confusion. I am pretty sure that the method I adopt does not result in the conclusion that cancer generates a jaw-dropping 6 billion bits of functional information. If it does, then I will immediate abandon this approach. I am also highly sceptical that cancer can produce even 300 bits of functional information. We shall see.
One more thing … I appreciate the kind of discussion we can have on this forum under the rules that Joshua has laid out. It permits us to rigorously, carefully, and even pedantically examine my approach to functional information. If one is allowed to gloss over certain things, flaws can be “hand-waved” over, and discussion of the theory becomes like trying to nail jello to the wall. None of us want that, and my initial impression is that this is the kind of forum where I can carefully defend my approach like I would in a thesis defence. Science can be a bit pedantic at times, so I hope you will be patient. Joshua has suggested that his paper on cancer will be an opportunity to test my approach, and I very much look forward to doing that once we understand my methodology and the theory behind it.
Here is a brief history of the approach I take, and I ask that you would do me the favour of reading through the entire account in an effort to better understand the basis for my approach.
I cannot claim that this is “my” approach: I saw a comment in one of the other parallel discussions, speculating that I “took up” some ideas presented by a Robert Marks and/or some others. That is not true. Although I’ve seen some of their papers, I have not taken the time to read them carefully and cannot claim to be familiar with their work. Thus, I am in no position to attempt to build on what Marks has done.
I began to take in interest in the application of information theory to biology long before I ever heard of “intelligent design”, Dembski, or Marks. What got me started was a paper published by Leon Brillouin in the Journal of Applied Physics in 1951, that I came across in the late 80’s titled, ‘Physical Entropy and Information. II’. Brillouin, as you are probably well aware, became the primary contributor to information theory immediately after the publication of Shannon’s famous paper in 1948. Much of Brillouin’s 1951 paper deals with the relationship between physical entropy and information and he ends up defining the relationship in terms of ‘negentropy’,
I = -∆ S = ∆ N , S , entropy; N , negentropy (which he denotes as Eq. 53).
I was always uncomfortable with trying to relate physical entropy to information, although the mathematical descriptions are similar. Consequently, I merely regarded his concept of negentropy as interesting, but that is all. He then went on in his paper to discuss information and, specifically, Shannon information in sections VI and VII, but he states something in the final section IX that got me started in the area of functional information, particularly because he discussed the idea of constraints (which later figure heavily in discussions of functional information). He wrote,
"The real physical entropy of the system is very much larger than the physical entropy with constraints. The negentropy,
N ( A ) = ( S ( r phys) – S (phys)), (57)
represents the price we have to pay in order to obtain a readable’ message, which our receiving station can interpret correctly."
His N ( A ) = I = -∆ S = ∆ N = - ( S ( r phys) – S (phys)), where S ( r phys), represents the entropy with constraints. Ignoring the concept of negentropy N we get,
I = -∆ S = - ( S ( r phys) – S (phys))
or
I = ( S (phys) - S ( r phys)) (I’ll denote this as Eq. 1a, because it is fundamental to everything else I discuss and I want to distinguish it from Eq. 1 in my earlier post)
This is known as Kullback-Leibler divergence (K-L ) or “information gain”. Thus it is not to be confused with mutual information, conditional information, or joint information. What was important here was his mention that to get a “readable” message, certain constraints would be required in the physical system. To my mind back in the late ‘80’s, there was a direct application to the problem of the biopolymeric sequences digitally encoded in the genomes of life, and the constraints on the digital information imposed by, ultimately, the laws of physics coupled with the desired function, if we desired to get stable, functional 3D structures in proteins.
Before I ever heard of Marks, Dembski, Johnson, or “intelligent design” I began to give seminars at various universities, and possibly a few in the US, beginning in the 1980’s on the application of what I called at the time ‘brillouin’s equation’ to functional protein sequences. My terminology evolved into discussing it in terms of “functional entropy” or “functional uncertainty”, with a nod to Shannon’s terminology. I submitted a paper to the Journal of Theoretical Biology around 1992, and it was sent out for review. One of the reviewers responded with an wrath-filled, 8-page, 10-font rant. It appears he/she was vehemently opposed to the idea that DNA encodes digital information, supposing that if it actually did there would be theological implications. His/her rock-solid belief, therefore, was that information theory applied to biology is nothing more than “theology” and seethingly recommended to me and the editor that I submit the paper to a theology journal, despite the fact it had absolutely zero mention of, nor any reference to, anything related to theology and it was filled with nothing but math and genetics. This was before the days of Google and intelligent design, so it showed that, back then, there were scientists highly hostile to the idea of biological information and who were astute enough to sense there might be theological implications. I’ve no idea who the reviewer was, but I was sufficiently demoralized to not bother submitting it anywhere else.
Fast forward to 2003 and Jack Szostak’s short article in Nature . When I read the article I was mildly shocked. The content of that article was virtually identical to the content of the lectures I had been giving for ten years previous. I considered sending him an email asking if he had ever heard one of my lectures, but decided against it. I wanted to avoid any suggested accusation of plagiarism, realizing that what both he and I were talking about seemed so easily arrived at each on our own. The problem was, using Hazen’s later (2007) M ( E ( x )), the actual value for M ( E ( x )) is an unknown for protein families. However, with the advent of online databases, specifically Pfam, I began to have the kind of data that might enable me to estimate the functional information required for protein families so I published the above linked-to paper which came out the same year as Hazen’s (though I had no knowledge of it when I first submitted my paper in January of that year. In this forum, we will test out the method I have adopted, and use Joshua’s cancer paper in that process, though I must first address some of the initial remaining concerns he has.
My day is filled with meetings tomorrow and I have some lecture prep I’m a bit behind in, so it may be Wednesday or Thursday before I can take the time to post again. If there is a delay in my responding, I have not disappeared; I will return! 