Carlo,

Great essay. Entropy has been considered as the number of ways one can rearrange microstates in such a way that a macrostate is preserved. Information I_n = log(P_n), for the occurrence of a state with probability P_n, when summed over P_n is

S = -k sum_n P_nI_n.

For P_n = 1/N one gets the standard result that S = k log(Ω), with small errors. In this case the system with equiprobable microstates is at maximum entropy. The maximum entropy is hard to define, but a black hole is probably as close as we can get. It is entropy defined for a maximum number of degrees of freedom or qubits on an event horizon or holographic surface bounding a volume.

A system at maximum entropy can only be assessed to have such entropy if some measurement is performed, say of temperature with T = ∂E/∂S, which necessitates some probe be coupled to the system. This means there is another system with some Hamiltonian that couples in with the thermal system of interest. In this case there is a change in entropy where in the adiabatic limit usually means δS \lt\lt S. This would then imply that a relative entropy exists with a small change in entropy.

Cheers LC

Professor Rovelli,

I found your essay exceptionally instructive even though I am a decrepit old realist. May I please make a comment about one sentence you used in your essay?

You wrote: "If we have measured a system, the information we have about a system allows us to predict its future." It is my contention that one real unique Universe is eternally occurring in one real here and now, once.

Respectfully Professor, unique cannot be measured. Unique cannot be systematic. Unique cannot be informed or conformed or deformed. Unique cannot be allowed or denied. Unique cannot be predicted.

    Dear Carlo Rovelli,

    I've long thought about the relationship between subjective information and objective quantities relating to physical systems---the sun's heat, in your example. Your idea, I think, clears up that mystery completely, and it goes to show that there are still simple, but powerful answers out there to be found, of which you've provided one.

    I've also been interested in your reconstruction of quantum mechanics for a long time. I've thought about the idea that Bell inequalities/the Kochen-Specker theorem/etc., are ultimately related to the fact that one cannot find a single (ordinary) probability distribution to cover all experimental predictions of QM, especially all correlations between observables---that is, there is no probability distribution such that its marginals reproduce all experimentally obtained probabilities. Unless, of course, you allow the distribution to take on negative values, generalizing to a pseudo-probability; then, you get something like the Wigner function on phase space. I keep thinking that this ought to be related, in some way, to your approach, but so far I haven't thought about it enough. Also perhaps of interest is Kochen's recent reconstruction of quantum mechanics along similar lines.

    Anyway, thank you for posting this essay, it was a pleasure to read,

    Jochen

      Dear Joe Fisher. Yours is an interesting idea, I think around the Mediterranean it goes back to Parmenides. There is one thing I have never understood about this idea: if the Universe is so happening only once and in such an unique and undifferentiated fashion, how can you and I be so talking and telling things one another, in a manner that could alter our thoughts?

      Thanks for your insightful essay. I am wondering what the status of single molecule thermodynamics is in your approach, e.g. in the Szilard engine and the derivation of Landauer's principle. Clearly, a single molecule interacts with thermometers and the box walls in a very different way from a gas with a large number of molecules. It has no effect on them except on those rare occasions when it hits them causing a tiny fluctuation. Yet we are still able to use Gibbsian thermodynamics to reason about this system. Do you accept such arguments in your approach and, if not, do you have an alternative derivation of Landauer's principle or an alternative resolution of Maxwell's demon.

        I read quickly your article, and it is interesting.

        I am thinking that if it is possible to have a bijection between each statistical system and a thermodynamic system (for example the blind child and a thermodynamic process), then it is possible to associate to each statistical system some physical quantity, and it is possible to apply the law of thermodynamic.

        So, if there are two boxes, with red balls (hot molecules) and white balls (cold molecules), and a blind child (no physical measure) that make a random choice for each time step, then asymptotically the system is in equilibrium; if we write the second law of thermodynamics (in equilibrium point) for the blind child, then we must write that is impossible the passing of N red balls (for example 20) from a boxe to an other boxes (Clausius say that the entropy fluctutations are impossible).

        I can understand the impossibility of the perpetual motion: it is impossible the infinite reproduction of the same sequence of extractions from the two boxes, but I think that the Clausius statement is so weak.

        Ciao Carlo,

        This is a nice statement of your ideas. Two questions:

        1) It seems that on your account systems somehow "know" which degrees of freedom are going to be involved in their interaction. When you write, "If the coupling is such that it depends only on certain macroscopic variables of the gas, then the physical interactions of the gas and this system are objectively well described by thermodynamics", I feel that the coupling of systems somehow precedes its physical description. Who tells us which coupling is correct and which isn't?

        2) On Democritus. I leave aside Democritus's argument about infinitely divisible space and no smallest magnitude, which your position obviously contradicts. Not everything in Democritus is the historic source of your account. However there's a contradiction that appears to me subtler than this. Democritus famously claimed that atoms can be any size, even the size of the Universe; they don't have to be small, like in Epicurus. This is good news, because you can favorably compare Democritus's notion of an atom and your notion of a system. Now, Democritus's systems are indivisible - this is absolutely fundamental for an atom. But aren't systems divisible? We routinely speak about subsystems, and this is as fundamental for quantum mechanics (e.g. entanglement) as indivisibility is for Democritus. So the analogy seems to fail on at least one very important point.

        Hope to see you around soon.

        Amicalement,

        Alexei

          Yes, it is a wonderful phrase. Of course is not me: it is Democritus...

          Hi Alex. Thanks for the comments. Here are answer to both:

          1) I was not sufficiently clear. The point I am trying to make is that the relevant physical question is not which coupling is correct and which coupling is not correct. The question is which coupling happens to *actually* be there, and which does not. The *physically relevant* coarse graining is the one determined by the couplings of the system with a second system that actually exists; and it is physically relevant because it describes the interactions of the system with this second system. Therefore there is nothing to "know", or "choose".

          2) Regarding Democritus, I spent the last year trying to bring together and understand what he might have said. I do not think he had any argument *for* infinitely divisibility of space. The other way around, I think that we can gather from what Aristotle says about him that he had argument *against* the infinite divisibility of anything. Regarding the fact that there are very large atoms, the attribution to him of this idea idea is widely disputed by many scholars. It is likely that it is a misunderstanding. In any case, I definitely not identify atoms with systems, for many reasons, including the one you say. Also, of course I am not defending Democritus naive physics here!

          Thanks for pointing out this. I do not know the answer. I will think about the Szilard engine and the derivation of Landauer's principle. c

          Carlo, it's good to see you bringing your expertise into the system. I like the idea of using Shannon's relative entropy to resolve the difference between Democritus and Plato/Aristotle.

          Since you are interested in ancient philosophy have you heard much about the acataleptic philosophers? There does not seem to be much recorded and interpretations differ.

            I suppose this word refers mainly to Pyrrho. His skepticism has been taken into the modern era by Hume, and is still alive and kicking, for instance in Popper. I take it to make the point that we are never sure, a good point. But what we care is reliability, not certainty. This is what I think (reliably, but not with certainty.)

            In the essay I mentioned Carneades and Arcesilaus rather than Pyrrho. What was of interest to me was the debate around whether a skeptic could say anything reliable about their own ideas if they did not consider anything reliable. Hume was more important in my previous essay on causality. At least the ideas are linked.

            I think a better modern development might be Bayesian Probability

            Carlo,

            I found your essay extremely interesting and informative, rich and diverse. I like the closing paragraph, particularly "The world is not just a blind wind of atoms, or generally covariant quantum fields. It is also the infinite game of mirrors reflecting one another formed by the correlations among the structures formed by the elementary objects".

            All the best,

            Antony

              Carneade... chi era costui?

              (Sorry, this is a joke only Italians understand ...)

              Hi Carlo,

              Your essay very worth reading..., I also love your analogy of the box full of balls, characterized by color and charge. Nothing much to criticize but you may wish to ponder the following to which I have not got any satisfactory answer.

              1. Since your essay touches on thermodynamics, given the equation

              dS = dE/T, if you are given a tiny amount of energy, E and a control knob that can regulate temperature, T, can you cause an astronomical-sized increase in entropy, S by manipulating the temperature, T at the time of energy introduction?

              I asked, Anton Biermans, but didn't get a satisfactory answer. I am looking at a cosmological implication.

              2. On the discretization of space, which you also touched on... I have asked Edward Fredkin, but I am not fully satisfied with the answer and Stephen Wolfram didn't respond.

              It is easy to say planets, air, fish in water, the water itself, atoms, etc, are discrete. Space does the separation for us so that we are able to call them "discrete".

              But when the great SEPARATOR itself, space is said to be capable of taking a discrete form, who will do the separation for us? Certainly, the separator cannot separate itself or can it? That is between one discrete representation of space and the adjoining one, what makes us distinguish 1 from 2?

              3. Then although you mention the atomism of Democritus, I suggest that you consider monads as well in future. Here is what Leibniz has to say about them, "... something that has no parts can't be extended, can't have a shape, and can't be split up. So monads are the true atoms of Nature--the elements out of which everything is made".

              I will be happy to have your expert opinion and criticism of my essay, where I discuss my ideas on how discrete nature of space can be realized.

              Regards,

              Akinbo

                Respectfully Professor Rovelli,

                Unique is different once. It cannot be undifferentiatedly fashionable.

                Joe

                Dear Carlo,

                A concept that information is a relative phenomena goes against classical and intuitive view of the world, to the point that even well-meaning grads still try to find a "universal underlying information", were none is present. Not further then one month ago we had such conversation, in two long batches of posts in this very competition at http://www.fqxi.org/community/forum/topic/1597 with Jochen. My use of magic words "Relational Quantum Mechanics" did not trigger at all that "all information is relative", or "there is no underlying universal information/reality", however obvious it may be in my own mind. Yet, in a longish back-and-forward, in a second batch, we came to a little formalism for highlighting the relative nature of information, and, therefore, descriptions. It seemed useful in driving information-relativity point. We even used Schmidt decomposition to provide a bridge between different informational perspectives.

                Reading your paper from 1996, which is a 2nd reference in your essay, I like your "... keep in mind that the observer can be a table lamp". I am using in discussions a qubit as an observer, the "smallest" observer in informational sense.

                In the 96's paper you had to tame notions of system and system's state, clarify and expose ambiguities. In the essay http://www.fqxi.org/community/forum/topic/1597 we make a guess of what is there in reality, taking cues from seemingly infallible QFTs of Standard Model. Then a notion of system becomes effective, and restricted by initial definitions, removing many common problems (e.g. cannot have priviledged systems, making all same). Then, we ask a question about information flow between interacting and non-interacting systems, which is settled by looking at exepriment, leading to "interaction confinement" concept. Interaction confinement automatically translates into unitary evolution of closed system, and implicitly shows that description and information are relative, using your more exact words. I wonder how we may combine these concepts better to further the common cause for having even smoother description of QM phenomena.

                Cheers,

                Mikalai.