Relative information at the foundation of physics by Carlo Rovelli

Carlo Rovelli

Essay Abstract

I observe that Shannon's notion of relative information between two physical systems can effectively function as a foundation for statistical mechanics and quantum mechanics, without referring to any subjectivism or idealism. It can also represent the key missing element in the foundation of the naturalistic picture of the world, providing the conceptual tool for dealing with its apparent limitations. I comment on the relation between these ideas and Democritus.

Author Bio

Carlo Rovelli is professor of theoretical physics at the University of Aux-Marseille. His main interest is quantum gravity, but he has worked also on the foundations of quantum theory and general covariant statistical mechanics, and on the ancient history and philosophy of physics.

Download Essay PDF File

Edwin Klingman

Dear Carlo Rovelli,

Thanks for submitting a very interesting essay. As I understand it, you begin by depicting entropy as emergent, which is then followed by "entropy is information". You certainly managed to link the two, as did ET Jaynes in 1957 when he presented the first information theory treatment of statistical mechanics. Yet he makes clear (quoted in my technical endnotes) that information entropy and thermodynamic entropy are not identical. In this respect I find your treatment of information as implying physical constraints on two interacting systems to be quite insightful.

Your discussion of information and quantum theory is succinct, but I was particularly pleased with your formulation that: "it is always possible to acquire new information about a system", thereby making the previous quantum information irrelevant. Very interesting.

Your summary of the relevance of information to life (i.e., to anything other than a dead universe) is fascinating and yet, if I understand your essay, you do not posit It from Bit, which is the question the essays address.

As a key player in quantum loop gravity I do not expect you to agree with my essay, but I still think you might enjoy a fresh perspective, and would hope you will read it and comment on it.

Thanks again, and good luck in the contest.

Edwin Eugene Klingman

Carlo Rovelli

Thank you Edwin for your very insightful comments. Indeed Jaynes views have affected me strongly (and I should have cited him!); but I found that there was still some bits missing in his account, and I am searching the missing element in Shannon's clean definition of relative information.

I certainly now will go read your essay.

best, carlo

Wilhelmus de Wilde

Dear Carlo,

It is a pleasure to see that after 3 years you now entered with a exeptional essay.

I cannot but totally agree with your perceptions. Especially the phrase "an infinite game of mirrors reflecting one another" is a wonderful expression.

You also put limits to our so called "material universe" as we perceive it.

"The "particular configurations of atoms singled out because of the manner a given other system interacts with them" allows me to think that one of these systems is our consciousness.

The latest contribution of your co-creator of LQG Abhay Ashtekar mentioning that there cannot be a singularity in a black hole but that at the Panck-length we enter in a "different" dimension is actually paralel with my perception of the limit of our causal universe, only I use the term "Total Simultaneity" and "rimal Sequence" in my essay : "THE QUEST FOR THE PRIMAL SEQUENCE" (which I hope you will read and rate).

The limit however as Ashtekar put in a Black Hole, is in my perception at every point in our causal universe attainable, why only in a black hole ?

Wilhelmus

Jacek Safuta

Dear Professor Rovelli,

I am familiar with your publications for many years mainly concerning LQG. It is a pleasure to welcome so great physicist here as an entrant to the contest.

Your essay delivers a new interesting look for the thermodynamical entropy (at least for me).

However I would like to comment your example with colored and charged balls that seems to be unfortunate. I would say the entropy of this isolated system cannot increase or decrease. The balls are static in a box because of the gravity force. We shall not assess the entropy of the system in this case as low or high. There is no dynamics/evolution here. A change is not possible as far as no external force will be added to the system. I think that entropy is a quantity that can be assessed only for an evolving system.

As I understand well in your interpretation entropy is the amount of information needed to specify the physical state of a system in relation to the state of another system. I think the role of thermodynamic entropy is first of all to understand how and why that information changes as the system evolves (as far as possible). My first choice is self-organized critical system that is statistical system naturally evolving without fine tuning to critical states in which correlation functions are scale invariant (referring to LQG). DNA code is a form of that evolving information (SOC) . I am working on a concept that applies the Darwinian evolution beyond its original sphere of organic evolution on Earth. Extremely shortly I propose the conformally flat spacetime to be the ancestor to the spacetime deformations i.e. wavepackets (particles and fields being these wavepackets - some form of geometrodynamics by not Wheeler's one) .... and at last up to DNA. Ad hoc I would name the concept the evolutionary geometrodynamic. Someone could name that concept a fantasy but I have proposed a spin experiment that delivers a falsifiability to it. So it generates obvious predictions.

It would be a great honor for me if you could take a look at my essay http://fqxi.org/community/forum/topic/1609 and the experiment and maybe comment it. The experiment description can be found only in references and I have some notices from the other entrants e.g. Torsten Asselmeyer-Maluga that the description is not quite clear so I could deliver an explanation if needed. My goal in the contest is not to win but to gather some new sometimes crazy ideas and some comments on my concept.

In your essay we can find three worlds: Plato's, Aristotle's and Democritus's. In my essay there are similar three worlds: It, Bit and Reality.

Best regards and thank you in advance.

Lawrence Crowell

Carlo,

Great essay. Entropy has been considered as the number of ways one can rearrange microstates in such a way that a macrostate is preserved. Information I_n = log(P_n), for the occurrence of a state with probability P_n, when summed over P_n is

S = -k sum_n P_nI_n.

For P_n = 1/N one gets the standard result that S = k log(Ω), with small errors. In this case the system with equiprobable microstates is at maximum entropy. The maximum entropy is hard to define, but a black hole is probably as close as we can get. It is entropy defined for a maximum number of degrees of freedom or qubits on an event horizon or holographic surface bounding a volume.

A system at maximum entropy can only be assessed to have such entropy if some measurement is performed, say of temperature with T = ∂E/∂S, which necessitates some probe be coupled to the system. This means there is another system with some Hamiltonian that couples in with the thermal system of interest. In this case there is a change in entropy where in the adiabatic limit usually means δS \lt\lt S. This would then imply that a relative entropy exists with a small change in entropy.

Cheers LC

Joe Fisher

Professor Rovelli,

I found your essay exceptionally instructive even though I am a decrepit old realist. May I please make a comment about one sentence you used in your essay?

You wrote: "If we have measured a system, the information we have about a system allows us to predict its future." It is my contention that one real unique Universe is eternally occurring in one real here and now, once.

Respectfully Professor, unique cannot be measured. Unique cannot be systematic. Unique cannot be informed or conformed or deformed. Unique cannot be allowed or denied. Unique cannot be predicted.

Carlo Rovelli

Dear Joe Fisher. Yours is an interesting idea, I think around the Mediterranean it goes back to Parmenides. There is one thing I have never understood about this idea: if the Universe is so happening only once and in such an unique and undifferentiated fashion, how can you and I be so talking and telling things one another, in a manner that could alter our thoughts?

Joe Fisher

Respectfully Professor Rovelli,

Unique is different once. It cannot be undifferentiatedly fashionable.

Joe

Jochen Szangolies

Dear Carlo Rovelli,

I've long thought about the relationship between subjective information and objective quantities relating to physical systems---the sun's heat, in your example. Your idea, I think, clears up that mystery completely, and it goes to show that there are still simple, but powerful answers out there to be found, of which you've provided one.

I've also been interested in your reconstruction of quantum mechanics for a long time. I've thought about the idea that Bell inequalities/the Kochen-Specker theorem/etc., are ultimately related to the fact that one cannot find a single (ordinary) probability distribution to cover all experimental predictions of QM, especially all correlations between observables---that is, there is no probability distribution such that its marginals reproduce all experimentally obtained probabilities. Unless, of course, you allow the distribution to take on negative values, generalizing to a pseudo-probability; then, you get something like the Wigner function on phase space. I keep thinking that this ought to be related, in some way, to your approach, but so far I haven't thought about it enough. Also perhaps of interest is Kochen's recent reconstruction of quantum mechanics along similar lines.

Anyway, thank you for posting this essay, it was a pleasure to read,

Jochen

Carlo Rovelli

Thank you! Carlo

Matthew Leifer

Thanks for your insightful essay. I am wondering what the status of single molecule thermodynamics is in your approach, e.g. in the Szilard engine and the derivation of Landauer's principle. Clearly, a single molecule interacts with thermometers and the box walls in a very different way from a gas with a large number of molecules. It has no effect on them except on those rare occasions when it hits them causing a tiny fluctuation. Yet we are still able to use Gibbsian thermodynamics to reason about this system. Do you accept such arguments in your approach and, if not, do you have an alternative derivation of Landauer's principle or an alternative resolution of Maxwell's demon.

Carlo Rovelli

Thanks for pointing out this. I do not know the answer. I will think about the Szilard engine and the derivation of Landauer's principle. c

Domenico Oricchio

I read quickly your article, and it is interesting.

I am thinking that if it is possible to have a bijection between each statistical system and a thermodynamic system (for example the blind child and a thermodynamic process), then it is possible to associate to each statistical system some physical quantity, and it is possible to apply the law of thermodynamic.

So, if there are two boxes, with red balls (hot molecules) and white balls (cold molecules), and a blind child (no physical measure) that make a random choice for each time step, then asymptotically the system is in equilibrium; if we write the second law of thermodynamics (in equilibrium point) for the blind child, then we must write that is impossible the passing of N red balls (for example 20) from a boxe to an other boxes (Clausius say that the entropy fluctutations are impossible).

I can understand the impossibility of the perpetual motion: it is impossible the infinite reproduction of the same sequence of extractions from the two boxes, but I think that the Clausius statement is so weak.

Alexei Grinbaum

Ciao Carlo,

This is a nice statement of your ideas. Two questions:

1) It seems that on your account systems somehow "know" which degrees of freedom are going to be involved in their interaction. When you write, "If the coupling is such that it depends only on certain macroscopic variables of the gas, then the physical interactions of the gas and this system are objectively well described by thermodynamics", I feel that the coupling of systems somehow precedes its physical description. Who tells us which coupling is correct and which isn't?

2) On Democritus. I leave aside Democritus's argument about infinitely divisible space and no smallest magnitude, which your position obviously contradicts. Not everything in Democritus is the historic source of your account. However there's a contradiction that appears to me subtler than this. Democritus famously claimed that atoms can be any size, even the size of the Universe; they don't have to be small, like in Epicurus. This is good news, because you can favorably compare Democritus's notion of an atom and your notion of a system. Now, Democritus's systems are indivisible - this is absolutely fundamental for an atom. But aren't systems divisible? We routinely speak about subsystems, and this is as fundamental for quantum mechanics (e.g. entanglement) as indivisibility is for Democritus. So the analogy seems to fail on at least one very important point.

Hope to see you around soon.

Amicalement,

Alexei

Carlo Rovelli

Hi Alex. Thanks for the comments. Here are answer to both:

1) I was not sufficiently clear. The point I am trying to make is that the relevant physical question is not which coupling is correct and which coupling is not correct. The question is which coupling happens to *actually* be there, and which does not. The *physically relevant* coarse graining is the one determined by the couplings of the system with a second system that actually exists; and it is physically relevant because it describes the interactions of the system with this second system. Therefore there is nothing to "know", or "choose".

2) Regarding Democritus, I spent the last year trying to bring together and understand what he might have said. I do not think he had any argument *for* infinitely divisibility of space. The other way around, I think that we can gather from what Aristotle says about him that he had argument *against* the infinite divisibility of anything. Regarding the fact that there are very large atoms, the attribution to him of this idea idea is widely disputed by many scholars. It is likely that it is a misunderstanding. In any case, I definitely not identify atoms with systems, for many reasons, including the one you say. Also, of course I am not defending Democritus naive physics here!

Carlo Rovelli

Yes, it is a wonderful phrase. Of course is not me: it is Democritus...

Philip Gibbs

Carlo, it's good to see you bringing your expertise into the system. I like the idea of using Shannon's relative entropy to resolve the difference between Democritus and Plato/Aristotle.

Since you are interested in ancient philosophy have you heard much about the acataleptic philosophers? There does not seem to be much recorded and interpretations differ.

Carlo Rovelli

I suppose this word refers mainly to Pyrrho. His skepticism has been taken into the modern era by Hume, and is still alive and kicking, for instance in Popper. I take it to make the point that we are never sure, a good point. But what we care is reliability, not certainty. This is what I think (reliably, but not with certainty.)

Philip Gibbs

In the essay I mentioned Carneades and Arcesilaus rather than Pyrrho. What was of interest to me was the debate around whether a skeptic could say anything reliable about their own ideas if they did not consider anything reliable. Hume was more important in my previous essay on causality. At least the ideas are linked.

I think a better modern development might be Bayesian Probability

Carlo Rovelli

Carneade... chi era costui?

(Sorry, this is a joke only Italians understand ...)

Antony Ryan

Carlo,

I found your essay extremely interesting and informative, rich and diverse. I like the closing paragraph, particularly "The world is not just a blind wind of atoms, or generally covariant quantum fields. It is also the infinite game of mirrors reflecting one another formed by the correlations among the structures formed by the elementary objects".

All the best,

Antony

Carlo Rovelli

Thanks!

Antony Ryan

My pleasure Carlo,

Hopefully you get chance to read my essay based around the Fiboanacci sequence and Black Holes.

Best wishes for the contest,

Antony