The Universe Is Not Like a Computer by Lorraine Ford

That wasn't very clear. When I concatenated the sentences, I took the result to be: "Hello. How are you today? Lovely weather, don't you think?". In other words, I forgot to say that I put spaces in-between the sentences as I was concatenating them. It does make a bit of a difference when working out the final information content per symbol (S = 2.8 with the two extra space characters, S = 2.9 without them), and so I apologize for making the mistaken of not specifying this in the original message.

Hi, Robert,

"The point that you have missed is that Information Theory is all about how inanimate entities, like radio receivers, deal with "information". Hence, its definition for "information" has nothing whatever to do with what more complex entities, like humans, subsequently do with that information. On the other hand, in a reductionist sense, it has a great deal to say about the behaviors of all the inanimate entities from which humans are constructed. In that sense, it says a great deal about human behaviors."

That, of course, is traditional Shannonian Information Theory you're talking about, which doesn't address "meaning" instead limiting itself to the abstracted transmission of physical data. (And always factor a few subtle redundancies into your writing ... you clearly do it very well and I like to try.) Anyway, there's long been an undercurrent of dissatisfaction with Shannon Information because it ignores the more subjective aspects of Information as that word is understood and employed in ordinary language. At last, however, we may have a New Testament: Jan KÃ¥hre's "Mathematical Theory of Information". It's pretty cool and addresses many received limitations but this isn't the proper forum in which to discuss it assuming I could.

Yo, Lorraine,

"Pure information" or "Information qua Information" is an intuition just as "Energy (qua Energy)" is an intuition. All that Qua stuff. We can observe Information and Energy only in their coded forms -- words on a page or in conversation, or sparks shooting out of a wall socket while you hope the circuit-breaker works so all that electrical energy won't decode into major thermal energy. ... Unless, possibly, isomorphisms can be taken as evidence of pure information. How come, for instance, wave mechanics is wave mechanics whether the waves in question are quantum or hydraulic? And speaking of double-slit experiments: There's at least one demonstrating that information can be erased and then made to reappear. But maybe that's too much about something.

Okay ... Jan Kahre. That neat "a" with the tiny "o" over it looked fine in the preview.

Apologies ahead of time if I'm being overly intrusive here, but I find this essay and these comments to be very fascinating, and I'd like to know more about what this is all about.

nmann, I think I see now what you mean by Shannon's measure of information does not explicitly account for meaning. In Shannon's measure, meaning is only implied -- each distinct symbol (message, datum, whatever we wish to label them as) implicitly has its own meaning, otherwise if each symbol had the same meaning there would be no information. Of course, this does not account for the meaning itself, kind of like saying that there's a implicitly a different meaning for 'red' and 'green' even though we don't go further to specify anything about photon wavelength (or whatever the deeper basis of photon wavelength may be).

So, am I right then that this search is about not only the information present in the whole symbol itself, but more so about counting up all of the information required to fully decode the symbol -- like, how the word 'the' all by itself as a single datum contains no information, but the dictionary definition of the word 'the' contains many distinct words and thus a non-zero measure of information? Of course, dictionary definitions are likely to be self-referential at the second level -- the definition of 'the' might contain the word 'it', and the definition of 'it' might contain the word 'the', thus creating an infinite loop. So, is this search then about finding not only a full-blown "definition" of reality, but one that contains no such infinite loops?

Hi Robert, Shawn, nmann, and Lorraine,

Re: "Shannon's measure of information does not explicitly account for meaning".

I am not at all certain that meaning can be quantified, and therefore that any meaning beyond Shannon's treatment is possible. Meaning is *always* contextual, as I indicated above with

"One if by land, two if by sea."

This is simply meaningless unless one knows the history [true or not] of Paul Revere's midnight ride, of April 18, 1775. And further that the numbers reference the number of lanterns to be shown in the steeple of the old North Church, the source of the information from observers who were watching to see whether the British Army was coming by land or sea. More meaning is attached to the fact that this preceded the Battles of Lexington and Concord during the American Revolution.

The context can be expanded indefinitely, as is, I believe, much of the point that Robert McEachern makes in his essay concerning the amount of initial data that is involved in interpreting any information, including even the simple 'up' or 'down' of the particle in an EPR experiment, based on sending 'entangled' particles through inhomogeneous magnetic fields, etc, etc, etc.

It is the inherent meaninglessness of physical data that calls the notion of information into question as in any way of physical import. At root is 'interaction' and if one 'source' interacts with another 'detector' through an intermediate medium or 'channel', then one can claim there is information transmitted. I'm not sure one can claim anything meaningful beyond this. Everything else is contextual and involves some sort of decoding or 'placing in context'.

Glad to see everyone focused on what is really a very important topic for physics today.

Edwin Eugene Klingman

Hi, S Halayka,

Some of what you're talking about sounds like it's relevant to the Newell-Simon theory in which meaning is claimed to reside in the symbols ("tokens") themselves instead of possessing the meaning a given culture or subculture assigns, often arbitrarily, to any particular symbol. According to Newell-Simon the programmed computer "reads" the symbol (token) then incorporates and processes the symbol's meaning without reference to any external context. Personally I don't buy that approach. Hilary Putnam (a comp. sci. pioneer, among other things) calls it "magical thinking." It dovetails also with what John Searle was getting at in his Chinese Room thought-experiment. Symbols mean nothing in and of themselves, even symbols fairly obviously (to us) representing the sun, a stick-figure person etc. They mean only what some collection of people agree they mean.

You could imagine a culture in which a symbol that looks to us like the sun meant something entirely different ... a god taking the form of a flower or whatever. Putnam has a riff about how he can't tell the difference between a beech and an elm and so he and someone else (who actually knows what an elm is) could appear to be talking about "elms" while actually talking gibberish because Putnam was thinking the whole time of beeches.

But Claude Shannon wasn't addressing that issue anyway. He was interested in how to transmit a given data-string (it could be anything at all) with minimum garbling, and ended up inventing mathematical information theory by identifying entropy as a link between information and thermodynamics. It's sort of Information Mechanics, a raw formalism. You want to transmit the word "dog" without "cat" being received at the other end. Maybe in the language you're employing "dog" means what we call "cat" and vise versa. That's not important.

For completeness I should have included the phrase:

'One if spin up, two if spin down'

to complement

'One if by land, two if by sea'.

So little meaning without the understood context.

Hi E.E.K.,

I went to lunch in the middle of composing my post and so we cross-posted. Offhand I don't see disagreement but we'll probably both find something.

The first sentence of my long post should have read something like:

"Some of what you're talking about sounds like it's relevant to the Newell-Simon theory in which meaning is claimed to reside in the symbols ("tokens") themselves instead of THE SYMBOLS possessing WHATEVER meaning a given culture or subculture assigns, often arbitrarily, to THEM."

More or less.

Hi Edwin and nmann,

Ok, so far I think I am on the right page insomuch that Shannon entropy is not what's being searched for here.

I really like that example of "one if by land, two if by sea". It was very informative for one, but it also helps me see meaning is a big can of worms here. The analogy of "one if spin up, two if spin down" is also great because it points back to Boltzmann and von Neumann entropy, which are pretty much analogous to Shannon entropy, if I'm not mistaken. Like with Shannon entropy, the problem with von Neumann entropy is that there is also meaning involved, and so not even this version of entropy is helpful. For instance, in terms of a black hole, someone might say that the entropy S means that there are "bits" which could represent a binary spin up/down which means that the event horizon area is quantized (and thus black hole mass is quantized), and another person (Edwin, myself) might say that it's more than just about binary spin and that the entropy is not discretized as such.

So, how would our description of physics change from this von Neumann approach? Is that what this search is for? To eliminate room for interpretation? I apologize if this is frustrating.

Would it be easier to discuss it in terms of object oriented programming, where there are classes (categories) and class member functions (morphisms that act on categories in a way that do not alter the fundamental structure of the category)? I know this kind of thing far better than I know physics.

I'm just having trouble visualizing what kind of "thing" would be in hand when the search is complete -- like what's its structure? I read one of those papers mentioned earlier about the Mathematical Theory of Information by Kahre, and it said right near the beginning that their theory gives Shannon entropy in a limiting case. I could not follow their argument, but it seems that they weren't eliminating Shannon entropy, but more like generalizing it. Help? Please? :)

- Shawn

Thank you for yet another reference to look up. I am reading it and trying to digest it all.

I suppose a better question is: what do you mean by 'information qua information'? Perhaps knowing that will help me see Lorraine's point of view better.

I apologize for taking this off on a tangent by bringing up "meaning", especially after Edwin had already pointed out that it was a can of worms. I just wanted to eliminate what's *not* beinig searched for. I do appreciate your patience with me.

Haha, I have to laugh at myself a little bit here. You and Edwin definitely bring up a good point about cultural differences -- even the word information is in and of itself context-specific. I think that this is why I am having trouble getting off of the ground here, because I have thought of the word information in terms of entropy for so long. Like, when someone says to me "So, what's new?" I think of them as saying "Please give me information.". I'm not trying to be intentionally slow-witted here, I'm just having some trouble reprogramming my brain. I do appreciate all the answers to the questions that I'm asking. Again, apologies if it's frustrating.

Regardless of whether or not this has anything to do with what you're trying to explain... it sure would be neat if we created a neural network that takes an input state and gives an output state, and by luck (huge luck, involving how we picked the training to take place) the network started predicting experimentally-verifiable results that our current rule-based theories didn't call for. :)

Well. I was hoping not to have to get into all this but...

Every coin has two sides. And so does Information Theory. Consider nmann's earlier quote, that "traditional Shannonian Information Theory you're talking about, which doesn't address "meaning" instead limiting itself to the abstracted transmission of physical data". Like a weird, quantum superposition, that statement is simultaneously true and false. It is true that is does not address the "final meaning", that a human will eventually slap onto the received message. But it very powerfully addresses a much more important meaning; one that enables the human to receive any message at all.

Here is the problem: Suppose you receive a message. That message has been corrupted. In addition to the information that the sender intends for you to receive, you are also receiving a great deal of information about all the things contributing to the corruption; distortion, multi-path interference, co-channel interference, doppler frequency shifts, time-varying amplitude attenuation, additive noise. You get the picture. It can be a really messy picture. So here is the problem; How do you know that all that "crap" is in fact "crap?" How do you know that that is not exactly what the sender intended you to receive?

If you know, a prior, that the sender is using only a very limited set of "channel coding symbols" (think of each as a uniquely modulated, short waveform), then anything that does not appear exactly as expected, must be corrupted. In the past thirty years, very powerful techniques have been developed to FORCE corrupted symbols to appear uncorrupted, by exploiting a priori information about these symbols.

This means that you know where the intended, as opposed to the unintended (corrupting) meaning lies.Think of this as always receiving two simultaneous messages, all mixed and garbled together, one containing the intended meaning, the other all the undesired, unintended meaning; and it does "mean" something; if you cared to, you might be able to learn a lot about the sources of the corruption, if you tried to analyze it. But we are not even going to try. It is the dirt, we are going to go for the gold.

In effect, such a system knows EXACTLY what it is looking for, and is trying very hard, often with very great success, to completely ignore all the "crap" that is known, a priori, to be of no interest. Have you ever wondered why your eyes cannot detect the absorption lines in the solar spectrum of visible light? That is why. Have you ever wondered why your auditory system cannot understand a fax transmission, sent via an acoustic modem over a telephone line? That is why.

So what does this have to do with physics? Well. If you make a guess about what the entity being observed "IS", then you might be able to exploit that information in the same way that the above channel coding can be exploited. For example, if you guess how many components "SHOULD" be measured, and you guess correctly, you may be able to extract a far "cleaner" observation than if you made no guess at all. But if you guessed wrong...

Rob McEachern

Shawn,

Never apologize! Never explain!

The "qua information" is maybe easier to get a handle on if you use "qua energy" as a paradigm. Nobody has ever observed Energy as such. But still we feel at home with the idea of Energy. We even have an intuitive sense of the concept Energy (or believe we do). It's like the feeling you get when you're running fast or taking off in an airplane. Or seeing horses galloping. Whatever. But we're not actually observing Energy as such. We're observing ourselves and other physical objects channeling energy and being energetic.

So back off and detach the sense of Energy from anything specifically energetic. You do that all the time. E=mc^2. That works as "pure energy." Even though in actuality you've only ever observed or sensed motion, heat, lightning ... coded energy.

Lorraine does something similar with Information, I suspect. She imagines, intuits something existing behind and prior to the coded messaging more generally associated with Information. Here's a link ... the most relevant material is the first part. I don't think Lorraine would agree with the general tenor of the paper, but it gives me a sense of what pure (or reasonably pure) Information might be like:

Quantum Physics as a Science of Information

Shawn, you say, "even the word information is in and of itself context-specific".

*Everything* is context specific. In fact, in an essay posted yesterday, "Cosmic Solipsism", Amanda Gefter states, "Instead, I argue, each observer has their own universe, which constitutes a complete and singular reality." While I would not go so far as solipsism, I think that she is right if she means that "each observer has their own MENTAL universe" or singular world-view. The initial genetic data that grew our unique brains and the information that shapes our learning brains could not possibly produce the same total picture. This is the beauty of math, and hence Shannon's treatment of information as data in a channel -- we abstract meaning from the symbols and treat them as meaningless entities, to which only logic applies. Once we move these from the world of math into a real physical universe in which every thing must [I assume] fit together in self-consistent fashion with everything else, then context [initial data] is all.

I believe that is what Robert McEachern is saying and I believe his essay one of the most important here.

I plan on taking no more of Lorraine's thread, but thanks all.

Edwin Eugene Klingman

Hi Robert and Edwin,

Thank you both very much for the further explanations. I have a basic idea of some error correction techniques used in TCP (checksums, acknowledgements, sliding window), and I'll use that as a kind of base to investigate more into what Robert is talking about.

I will definitely give Robert and Amanda's essays another read.

Thank you very much for taking the time to help me sort though this.

Ok, I think I'm catching your drift now. I can definitely see what you mean about how energy can be considered to be an intangible object that only makes itself known somewhat indirectly via motion. I think that some physicists say that there is an equivalence between energy and information (http://arxiv.org/abs/1207.6872), so perhaps this is 'qua energy' is very much one and the the same thing as `qua information'.

Thank you for the link to that paper about Quantum Physics as a Science of Information.

I will stop hijacking Lorraine's thread for now and read up some more before asking other questions.