Karl -
Congratulations on another excellent essay, this year. I agree that context is basic to how information works, and you make the point clearly and vividly. And it's a clever idea that context can be quantified just by looking at the same data in different situations.
Of course quantifying information was key to making it a useful concept in physics. But it's unfortunate that "meaning" still has no meaning there. The semantic aspect of information still gets treated as if it belonged to a separate realm of subjectivity and linguistics. Surely QM shows that physical information is at least as context-dependent as any other kind, if information only exists to the extent there's a context that can measure it. But as you say, how "information interacts with other information, on levels much higher than statistical analysis, can seem to be an opaque problem."
I've tried to focus on this in my essay, and explain why physical measurement-contexts are so hard to conceptualize. It's not that there's any mystery about what kind of context is needed for any specific type of measurement. It's rather that many different kinds of contexts are needed to observe different parameters, and that any measurement depends on other kinds of measurements, made in other contexts.
Your essay points out that a message can have many levels of meaning, depending on context. I think this is true even at the level of measurements, though we haven't yet learned how to sort out the levels. We still tend to think of "observing" as though it should be simple.
My suggestion is that the complicated web of different measurement-contexts is necessary for any information to be physically definable, and that the way to understand this kind of complexity is by looking at it in evolutionary terms. That is, information defined in one context gets passed on as a contribution to another context, where other information gets defined, etc. This process is very different from the reproductive process that underlies biological evolution... but I think we can understand it too as operating through a kind of natural selection.
Thanks for the nice writing, and for bringing a key issue into focus.
- Conrad