Well. I was hoping not to have to get into all this but...
Every coin has two sides. And so does Information Theory. Consider nmann's earlier quote, that "traditional Shannonian Information Theory you're talking about, which doesn't address "meaning" instead limiting itself to the abstracted transmission of physical data". Like a weird, quantum superposition, that statement is simultaneously true and false. It is true that is does not address the "final meaning", that a human will eventually slap onto the received message. But it very powerfully addresses a much more important meaning; one that enables the human to receive any message at all.
Here is the problem: Suppose you receive a message. That message has been corrupted. In addition to the information that the sender intends for you to receive, you are also receiving a great deal of information about all the things contributing to the corruption; distortion, multi-path interference, co-channel interference, doppler frequency shifts, time-varying amplitude attenuation, additive noise. You get the picture. It can be a really messy picture. So here is the problem; How do you know that all that "crap" is in fact "crap?" How do you know that that is not exactly what the sender intended you to receive?
If you know, a prior, that the sender is using only a very limited set of "channel coding symbols" (think of each as a uniquely modulated, short waveform), then anything that does not appear exactly as expected, must be corrupted. In the past thirty years, very powerful techniques have been developed to FORCE corrupted symbols to appear uncorrupted, by exploiting a priori information about these symbols.
This means that you know where the intended, as opposed to the unintended (corrupting) meaning lies.Think of this as always receiving two simultaneous messages, all mixed and garbled together, one containing the intended meaning, the other all the undesired, unintended meaning; and it does "mean" something; if you cared to, you might be able to learn a lot about the sources of the corruption, if you tried to analyze it. But we are not even going to try. It is the dirt, we are going to go for the gold.
In effect, such a system knows EXACTLY what it is looking for, and is trying very hard, often with very great success, to completely ignore all the "crap" that is known, a priori, to be of no interest. Have you ever wondered why your eyes cannot detect the absorption lines in the solar spectrum of visible light? That is why. Have you ever wondered why your auditory system cannot understand a fax transmission, sent via an acoustic modem over a telephone line? That is why.
So what does this have to do with physics? Well. If you make a guess about what the entity being observed "IS", then you might be able to exploit that information in the same way that the above channel coding can be exploited. For example, if you guess how many components "SHOULD" be measured, and you guess correctly, you may be able to extract a far "cleaner" observation than if you made no guess at all. But if you guessed wrong...
Rob McEachern