Essay Abstract

The Shannon-derived measure of surprisal, or the self-information of a message, is calculated relative to some contextual framework. Enriching context constrains a message's potential interpretation, typically enriching its information content in the process. This may have implications for a general informational theory in physics: The receipt of information by a system creates boundary conditions that constrain further new information, the receipt of which then imposes further boundary conditions, and so on. Such ever-tightening informational constraint, iterated over billions of years, may drive the evolution of complexity in an "it from bit" universe.

Author Bio

Although I have been a writer for most of my professional career, I have never left my science background behind (BA Biology, University of California Berkeley). I recently joined a tech startup that is working with members of Stanford University's physics and philosophy departments, developing what we hope will be the next big thing in education. My essay "Toward an Informational Mechanics" won a Special Commendation prize in FQXi's 2012 "Questioning the Foundations" competition.

Download Essay PDF File

Dear Sir,

Congratulations for presenting a brilliant paper in an authoritative and unambiguous style "in context", which is becoming rare in scientific circles. It has tremendous scope for generalization and extension.

Wheeler was right that the universe is informational. Information is specific data reporting the state of something based on observation (measurements), organized and summarized for a purpose within a context that gives it meaning and relevance (in context) and can lead to either an increase in understanding or decrease in uncertainty. Information is not tied to one's specific knowledge of how particles are created and their early interactions, just like the concepts signifying objects are not known to all. But it should be tied to universal and widely accessible properties. Thus, information is perceived as a concept about something. Since everything is constituted of smaller "bits", "It from Bit" applies both to objects and concepts.

Regards,

basudeba

Dear Karl,

Interesting essay... and I like your term "PRIMITIVE BIT", i.e. the answer to the most primitive question in an algorithm/ hierarchy right. What is the most "primitive" bit (answer) you can think of? Before this, what is the "most primitive question" you can ask?

Regards,

Akinbo

*You can check up my take on all this in my essay, "On the road not taken"

Basudeba and Akinbo -- Thank you for the kind words. Akinbo, the most primitive question that can be asked is "Yes or no?" And the answer to that question would be a primitive bit. Asking the question under any context whatsoever (i.e., yes or no about what?) enriches the significance of the answer (e.g., yes, there is a particle in this volume). Sometimes we refer to this significance as "meaning."

All indications seem to point to information being strictly relational, so there is no absolute primitive bit that we can point to. A primitive bit is simply a yes-or-no (or 0-or-1), relative to an observer or space that has no context. For a different observer, the same bit could be highly meaningful.

    Dear Sir,

    This can be explained in a different way also. Information is specific data reporting the state of something based on observation (measurements), organized and summarized for a purpose within a context that gives it meaning and relevance and can lead to either an increase in understanding or decrease in uncertainty. To be meaningful, the reporting of the result of measurement has to be perceived as such. Perception is the processing of the result of measurements of different but related fields of something with some stored data to convey a combined form "it is like that", where "it" refers to an object (constituted of bits) and "that" refers to a concept signified by the object (self-contained representation). This shows the relational aspect of information.

    How a communication is perceived? All sounds do not convey a message. When we say "pen" - a sound symbolizing three letters (symbols) arranged in a particular pattern, what is the content of the message for the receiver? To someone who can't hear or does not know English or have not seen a pen or knows the pen by some other name, the word "pen" or the object does not make any sense. If he has come across this word earlier and has known to relate the sound to the object, only then he can think that "It is like the one I had seen earlier, which was called a pen. Hence it is a pen". Thus the actual content of any word is the concept of a known object. This shows that information has two parts: the object about which information is being conveyed and the concept of the object as a distinct entity from its physical presence. This leads to matching or negative perception of the object and the concept.

    Regards,

    basudeba

    I think I understand what you're saying. But physicists generally don't want to discuss perception or words, or even meaning. So, I concentrated on a broader definition of information, something that reduces uncertainty for some system. I used the example of Morse code to illustrate how context can enhance uncertainty reduction for a human system receiving a human communication, but only for the purpose of suggesting that a similar principle may apply to all information in an "it from bit" world. -KC

      Dear Sir,

      You are right. They call perception as intelligence and meaning as collapse. But you cannot avoid word, as all concepts are expressed in words only. There is no other way of expressing a concept. You mat at best call it a string, but why not call a spade a spade?

      Regards,

      basudeba

      Hi Karl,

      Indeed, context can't be ignored, especially in quantum mechanics. The essay is well-documented, goes smoothly, and it is fun to read. Perhaps, if the space were not that limited, you could develop the phrase immediately before the conclusion. In particular, how "the accumulation of contextual constraint is local and in accordance with causality" in quantum mechanics, and how time gets an arrow from "accumulation of contextual constraint", when the very "accumulation" involves an arrow of time. Overall, nice essay!

      Best regards,

      Cristi

        Hi Karl,

        You have asked very important question: '...how can we build theoretical frameworks and models to test such a bold hypothesis as "it from bit"?' The majority of physicists seems to

        In my essay http://fqxi.org/community/forum/topic/1609 I have proposed not only a theoretical framework but also a simple experiment to test it.

        Best regards

        • [deleted]

        • Post #10

        Thanks Cristi. I wanted to keep my essay short and fairly limited this time. I see your point that the arrow of time idea is somewhat circular. Perhaps it would be better to say that if the universe is informational, then the arrow of time is practically trivial, a consequence of the sequence: information transmission --> reception --> complexification. So even if mechanical laws of physics work equally well backwards or forwards in time, information does not; it is unidirectional, and that directionality is what establishes the arrow of time. Perhaps the 2nd law is another "it-world" consequence of this directionality. Thanks for the thoughtful comment and good luck in the competition. KC

        I found your essay interesting. It is the case that standard information theory is just about the probabilities for occurrence of symbols and symbol strings. The standard idea of course is that the larger the state space, say dim(H) then entropy S ~ log(dim(H)) is a convex measure of the amount of information or "content." This of course tells us nothing about the meaning of this information.

        In my essay I look at causality according to modal logic, which is a representation for some logico-algebraic system that is equivalent to the transformation of qubits, qutrits, qudits or any quNit system. I illustrate how formal incompleteness of any structure of this type is not able to compute all possible states. This is related to Lob's theorem and this in other studies is involved with semantic soundness theorems. I can't say right now how this would impact the nature of your argument for a semantic entropy. Yet this might be something to look into.

        Cheers LC

          Thanks Lawrence. I suspect you're right that the question is ultimately undecidable. I look forward to reading your essay closer; I've admired your work here for several years. KC

          5 days later

          I spotted a potential source of confusion in my terminology. I distinguish contextual information from "new" information, e.g. at the top of page 5, where I use DNA as an example of new information. Obviously, there's nothing new about a DNA sequence. I should have used a more generic term, such as "primary" information, i.e., the information that is being interpreted in the presence of context. Thank you for reading. KC

          Dear Karl,

          You have again produced an excellent essay. I fully agree with your emphasis on context (and made many of the same points on Lorraine Ford's 2012 essay page). You note that "Wheeler looks for clues into how meaning emerges from bits", and that the "value" of information appears only in context, whether DNA interpreted by a cell or Morse code interpreted as 'SOS'. Thus the 'surprise' or information value is "not an absolute quantity" since it is context dependent.

          I agree with this and fail to see how such an entity can be a fundamental physics reality. Instead I view the process as one of energy exchange until a threshold is exceeded. If exceeding the threshold causes a change "in form" of the target system, then information is created and stored. The structure storing the information is the immediate local context, but may be only incidental to a larger context. In this regard I found Kuppers' statement about complexity very interesting.

          Also, you say "when old bits meet new bits they seem to multiply in some sense." How does this differ from Bayesian probability?

          You state that "the iterated interplay of ... information could create an evolving universe of increasing complexity." I can see how it creates a "model" of the evolving universe. It seems to me that all of your arguments work against information as a fundamental physical entity.

          It also seems that 'meaning' and 'context' demand consideration of conscious awareness. We are "aware of" meaning and "aware of" context. They have no strictly physical existence (although they may be 'modeled' in the physical structure of our brains). Therefore I cannot envision how a serious attack on problems of context can be made without bringing awareness into the picture. And I do agree with you that a serious attack on such problems is in order.

          I would be very interested in your response and in your response to my essay.

          Best regards,

          Edwin Eugene Klingman

            Edwin, thank you for the compliment, and for the challenging questions. In writing my essay, I found that I could not use the idea of contextual constraint to argue effectively either for or against "it from bit." So instead, after arguing that contextual constraint does happen, I tried to point out parallels between this kind of informational constraint and the more conventional physical constraint by physical law. My approach was more one of, "If the world is it from bit, then maybe this can shed light on how it emerges from bit."

            Statistically, there may be no difference between contextual constraint and Bayesian probability, at least in some cases. Perhaps the concept of Bayesian probability can be generalized to say something about the fundamental nature of the world. Sounds like a good essay thesis!

            The thing that motivates my essays is a search for ways in which the universe can be simpler than we think. The premise of "bit from it" works against that, in my opinion, because it requires almost a dualism, with both counterfactually definite objects and (presumably real) information about them. That's why "it from bit" grabs me -- because it calls for only the informational side, with space, time, and matter emerging in relation to informational subsystems such as ourselves. However, I chose not to go there in my essay.

            Thanks again -- I've glossed over all of the essays already and I am in the process of giving them close reads. Best of luck! KC

            Carl,

            Nice essay, thoroughly identifying the 'dumb simplicity' of binary data. I agree nature has a massive amount more to offer than the bit itself can tell us.

            You state; "Contextual constraint is ubiquitous in the physical world". I couldn't agree more, (indeed you may recall my last years essay where, as 'Propositions' in truth propositional logic all context had infinite context layers itself, and also infinite data subsets, for a hierarchical structure of Bayesian distributions between all assigned integers.

            Do you agree that this relation then gives a permanent layered situation where everything only has full meaning with respect to a local background.

            I wish I could restrain my own essay subject like yours. I develop that theme right through to some important implications for the 'hidden' information 'between' the simple binaries, or yes/no answers, and then describe how the concept in practice has the power to resolve the EPR paradox! Important, but probably far too dense again for most to stay with. I'd appreciate you view and comments on it.

            But well done for yours. Relevant, convincingly argued and entirely understandable.

            Best of luck in the contest.

            Peter

              Thank you Peter. After a couple attempts at more ambitious essays, for this one I followed the model of George Ellis' essay from last time. Here's someone who could write about any topic in physics or mathematics at any level, yet he chose to argue a narrow and straightforward thesis, simply by making observations from known science and tying them together. I can only hope to be as insightful.

              Regarding your question, I think the answer is yes, although I hesitate at the word "background." I think that for any informational system, its information is definable only relative to other local systems; that would be the local background. Members of the background, meanwhile, have their own relative local background. Absolute information does not exist.

              I'll be checking out your essay soon; thanks again and good luck. KC

              5 days later

              Karl,

              That agrees with the model I described last year; ("information is definable only relative to other local systems; that would be the local background. Members of the background, meanwhile, have their own relative local background.") and indeed the last 3! I identified it as an infinite hierarchy, which has the precise structure of 'propositions' in truth propositional logic. Each proposition can only be resolved wrt it's local background. It is equivalent to saying all inertial systems are real, but none is absolute.

              I took a break from that this year, but still tried to pack 3 essays in one! I build an ontological construction leading to an apparent 'unifying' resolution of the EPR paradox. It needs very careful reading. Do give me your view.

              best wishes

              Peter

              Dear Karl

              Very interesting to read, but a little hard to understand.

              http://fqxi.org/community/forum/topic/1802

              9 days later

              Dear

              Thank you for presenting your nice essay. I saw the abstract and will post my comments soon.

              So you can produce material from your thinking. . . .

              I am requesting you to go through my essay also. And I take this opportunity to say, to come to reality and base your arguments on experimental results.

              I failed mainly because I worked against the main stream. The main stream community people want magic from science instead of realty especially in the subject of cosmology. We all know well that cosmology is a subject where speculations rule.

              Hope to get your comments even directly to my mail ID also. . . .

              Best

              =snp

              snp.gupta@gmail.com

              http://vaksdynamicuniversemodel.blogspot.com/

              Pdf download:

              http://fqxi.org/community/forum/topic/essay-download/1607/__details/Gupta_Vak_FQXi_TABLE_REF_Fi.pdf

              Part of abstract:

              - -Material objects are more fundamental- - is being proposed in this paper; It is well known that there is no mental experiment, which produced material. . . Similarly creation of matter from empty space as required in Steady State theory or in Bigbang is another such problem in the Cosmological counterpart. . . . In this paper we will see about CMB, how it is generated from stars and Galaxies around us. And here we show that NO Microwave background radiation was detected till now after excluding radiation from Stars and Galaxies. . . .

              Some complements from FQXi community. . . . .

              A

              Anton Lorenz Vrba wrote on May. 4, 2013 @ 13:43 GMT

              ....... I do love your last two sentences - that is why I am coming back.

              Author Satyavarapu Naga Parameswara Gupta replied on May. 6, 2013 @ 09:24 GMT

              . . . . We should use our minds to down to earth realistic thinking. There is no point in wasting our brains in total imagination which are never realities. It is something like showing, mixing of cartoon characters with normal people in movies or people entering into Game-space in virtual reality games or Firing antimatter into a black hole!!!. It is sheer a madness of such concepts going on in many fields like science, mathematics, computer IT etc. . . .

              B.

              Francis V wrote on May. 11, 2013 @ 02:05 GMT

              Well-presented argument about the absence of any explosion for a relic frequency to occur and the detail on collection of temperature data......

              C

              Robert Bennett wrote on May. 14, 2013 @ 18:26 GMT

              "Material objects are more fundamental"..... in other words "IT from Bit" is true.

              Author Satyavarapu Naga Parameswara Gupta replied on May. 14, 2013 @ 22:53 GMT

              1. It is well known that there is no mental experiment, which produced material.

              2. John Wheeler did not produce material from information.

              3. Information describes material properties. But a mere description of material properties does not produce material.

              4. There are Gods, Wizards, and Magicians, allegedly produced material from nowhere. But will that be a scientific experiment?

              D

              Hoang cao Hai wrote on Jun. 16, 2013 @ 16:22 GMT

              It from bit - where are bit come from?

              Author Satyavarapu Naga Parameswara Gupta replied on Jun. 17, 2013 @ 06:10 GMT

              ....And your question is like asking, -- which is first? Egg or Hen?-- in other words Matter is first or Information is first? Is that so? In reality there is no way that Matter comes from information.

              Matter is another form of Energy. Matter cannot be created from nothing. Any type of vacuum cannot produce matter. Matter is another form of energy. Energy is having many forms: Mechanical, Electrical, Heat, Magnetic and so on..

              E

              Antony Ryan wrote on Jun. 23, 2013 @ 22:08 GMT

              .....Either way your abstract argument based empirical evidence is strong given that "a mere description of material properties does not produce material". While of course materials do give information.

              I think you deserve a place in the final based on this alone. Concise - simple - but undeniable.

              Karl -

              Congratulations on another excellent essay, this year. I agree that context is basic to how information works, and you make the point clearly and vividly. And it's a clever idea that context can be quantified just by looking at the same data in different situations.

              Of course quantifying information was key to making it a useful concept in physics. But it's unfortunate that "meaning" still has no meaning there. The semantic aspect of information still gets treated as if it belonged to a separate realm of subjectivity and linguistics. Surely QM shows that physical information is at least as context-dependent as any other kind, if information only exists to the extent there's a context that can measure it. But as you say, how "information interacts with other information, on levels much higher than statistical analysis, can seem to be an opaque problem."

              I've tried to focus on this in my essay, and explain why physical measurement-contexts are so hard to conceptualize. It's not that there's any mystery about what kind of context is needed for any specific type of measurement. It's rather that many different kinds of contexts are needed to observe different parameters, and that any measurement depends on other kinds of measurements, made in other contexts.

              Your essay points out that a message can have many levels of meaning, depending on context. I think this is true even at the level of measurements, though we haven't yet learned how to sort out the levels. We still tend to think of "observing" as though it should be simple.

              My suggestion is that the complicated web of different measurement-contexts is necessary for any information to be physically definable, and that the way to understand this kind of complexity is by looking at it in evolutionary terms. That is, information defined in one context gets passed on as a contribution to another context, where other information gets defined, etc. This process is very different from the reproductive process that underlies biological evolution... but I think we can understand it too as operating through a kind of natural selection.

              Thanks for the nice writing, and for bringing a key issue into focus.

              - Conrad

              Karl,

              If given the time and the wits to evaluate over 120 more entries, I have a month to try. My seemingly whimsical title, "It's good to be the king," is serious about our subject.

              Jim

              Hi Karl,

              I have yet to read an essay which treats the question where all information comes from, how information becomes information. What I mean is this: If there would be only a single charged particle among uncharged particles in the universe, then it wouldn't be able to express its charge in interactions. As it in that case it cannot be charged itself, charge, or any property, for that matter, must be something which is shared by particles, something which only exists, is expressed and preserved within their interactions. If particles, particle properties (its) are both cause and effect of their interactions, of the exchange of bits, if particles only exist to each other if and for as long as they interact, exchange information, then you cannot have one without the other nor can one be more fundamental than the other.

              If the information as embodied in particle properties and the associated rules of behavior a.k.a. laws of physics must be the product of a trial-and-error evolution, then information only can survive, become actual information when tested in practice, in interactions between its carriers, between 'its', between actual, physical, material particles. Part of the confusion originates in the supposition that particles randomly emit and absorb virtual photons and gravitons to communicate forces between them. If particles only exist to each other if and for as long as they interact, then in this view they would only exist, receive information about particles in their environment at the times they absorb a virtual particle, but would not exist to each other in the periods in between. This supposition in fact is the expression of the belief that particles, particle properties only are the cause, but not also the effect of their interactions, as if they would keep existing even when isolated from interactions, as if 'to be' is a passive state which requires no effort on the part of the particles, a noun instead of an activity, a verb.

              Regards, Anton

                Anton -

                As you and Karl both point out, a context of interaction with other things is needed for information about anything to be physically meaningful. I discuss the implications of this in my essay on The Evolution of Determinate Information.

                Thanks - Conrad

                10 days later

                Dear Karl,

                Contextual constraint is a great approach and deriving an arrow of time is always nice. My essay suggests another reason why we ought to have time behaving this way too.

                You say - that our universe is a "simplest-case scenario" and may contain far less information than is conventionally believed. This is very interesting, in conjunction with something I read earlier in another essay, where the author wrote that information increases over time while matter/energy remains constant.

                I think you've done a great job here.

                Well done & all the best,

                Antony

                23 days later

                Karl,

                After your kind message to me above ending;

                "I'll be checking out your essay soon; thanks again and good luck. KC"

                You seem to have dropped off the radar. I do hope you're well and managed to read it. I hope you'll comment and give me lots of points.

                I think your view on Contextual constraint and background information is very important and undervalued, as your score shows. Deadline due so hitting the booster now. Hope it helps.

                Very well done and thank you for helping me rationalise that point.

                Best wishes

                Peter