Spanspermia: Does Life Come from Outer Hilbert Space?

SepiaBeetle

Thanks again for the kind words. I really wish I had the time to engage with you properly on this many worlds point, but for now I'll just try and drop a provocative idea on your head, on the off chance it's illuminating. Would you believe in many worlds if the wavefunction was simply describing classical physics? That is, no intrinsic uncertainty, no entanglement between observable statistics, but an otherwise identical formalism?

I don't know how strong your mathematical background is, but if you're not afraid of some quite hideous algebra, I'd recommend taking a look at this paper: https://arxiv.org/abs/2302.13208. The question one should be asking themselves throughout is whether it would be a reasonable interpretation of probability to say every time we sample from a distribution, our observed outcome is actually just evidence that we are in an ensemble of all outcomes. Anyway that's about all I can sensibly say without breaking out a fair amount of formal machinery, but the fact one can make a formal correspondence to a classical system - and that this emerges in a singularity-free manner as the limit of commutativity - means that you have to wear classically all the interpretational baggage you lay on the quantum wavefunction. For me that renders MWI - and any non-statistical interpretation of the formalism - completely absurd.

EmeraldBeetle

SepiaBeetle Would you believe in many worlds if the wavefunction was simply describing classical physics? That is, no intrinsic uncertainty, no entanglement between observable statistics, but an otherwise identical formalism?... the fact one can make a formal correspondence to a classical system - and that this emerges in a singularity-free manner as the limit of commutativity - means that you have to wear classically all the interpretational baggage you lay on the quantum wavefunction. For me that renders MWI - and any non-statistical interpretation of the formalism - completely absurd.

Ah, you're one of those KvN people! Thanks for the refresher, and I know enough linear algebra to barely get me in trouble but it's not the proofs but their philosophical implications that interest me. Also WRT 'belief' in MWI, as a philosopher I don't believe in anything other than what my own empirical–phenomenal experience discloses, and any talk of external physical realities is by definition theoretical-model-speak. As the eminent game theorist Hugh Everett insisted, "any physical theory is a logical construct (model) ... some of whose elements are associated with elements of the perceived world".

But I've never quite understood why reformulating the mathematical formalism of Schrödinger's wave function and then taking the classical limit \hbar \to 0 in order to recover a Hilbert‑space representation of classical dynamics is anything other than a fancy‑dress maths trick. Nor how that trick is ontologically relevant to a quantum worldview.

If KvN could derive genuinely quantum interference phenomena – entanglement, Bell violations or environmental decoherence and pointer states – from the statistical interpretation of the formalism, then that's totally fine by me! We can all just forget about the 'quantum weirdness' that's been spooking people for the last century and reinstate a coherent “belief” in classical realism as the ontological form of the external physical reality. Quantum foundations done and dusted! But you can't, can you? It's an aspiration built on what I call 'classical nostalgia' for those simpler ontology days.

And as for those various 'belief systems' concerning the structure of the external physical reality, doesn't KvN simply presuppose classical phase‑space realism in its 'Hilbertisation' of classical phase space? My own intuition, following Kant, is that spacetime is an emergent perceptual reality of our phenomenal world and that we don't actually need to load that phenomenal spacetime‑grid “interpretational baggage” onto the external reality. That's why I find the simplicity of a universal wave function evolving as a single state vector in an infinite‑dimensional Hilbert space (with its noncommutative observable algebra) a rather intriguing philosophical model for the external reality. And Everett offers a bridge from there to our decohering, quasiclassical, local light‑cone phenomenal experience of our 3+1D phenomenal world.

The simplicity of that Everettian 'external picture' really does fit nicely with my neo-Kantian philosophical 'phenomenal picture', I guess much like your KvN formalism suits your neo-classical spacetime picture. And I can see how the former might appear absurd to you stubborn classicists, but each to their own belief system I say!

Lorraine Ford

EmeraldBeetle
What an unpleasant view to have of yourself and of other people, the view that you yourself are equivalent to a mechanical machine that is machine-interacting with other mechanical machine-equivalents!

SepiaBeetle

EmeraldBeetle

Thanks for the spirited reply. I see your problem perfectly, and you’re right, I’m "one of those KvN people"—at least insofar as the algebraic formalism provides a clear structure, which is the point of my essay! You've identified the crux of my problem with the Many-Worlds Interpretation (MWI) and other non-statistical views, but you've mischaracterized the purpose of taking the classical limit $\hbar \to 0$. It’s not about "classical nostalgia" or a "fancy-dress maths trick." It’s about consistency and interpretational solvency. The central point of demonstrating that you can formulate classical dynamics within an identical Hilbert space structure (the Koopman-von Neumann or KvN formalism) is simple: whatever interpretation you ascribe to the quantum formalism must be compatible with its classical limit.

If your interpretation (like MWI) claims the quantum wavefunction $\Psi$ is an ontological object—the real, evolving state of external reality—and that this reality splits every time a quantum measurement occurs, then that same ontological claim must hold true when $\hbar \to 0$ and the system behaves classically. Call this a principle of interpretational continuity, whatever you like. It's a view that's only recently become available because until recently the problem of the semiclassical limit's singularity was a discontinuity where you couldn't strictly identify one object as the limit of the other. The waveeoperator picture I linked now does that, and so an entirely new line of attack on interpretation becomes available. You are forced to wear the "interpretational baggage" (i.e., splitting worlds) in a purely classical setting as well. It becomes absurd to claim that the ensemble of outcomes when flipping a coin (a perfectly classical, commutative system) requires the existence of multiple, non-interacting realities just because the maths can be formulated that way.

The ability to smoothly recover the commutative classical limit means that you cannot introduce ontological claims about the wavefunction in the quantum regime that become evidently absurd in the classical regime. The classical world is just a special case of the quantum one (the limit of commutativity). If your interpretation requires "quantum weirdness" (like splitting worlds) to stop working precisely when the system looks classical, you are making an arbitrary, non-smooth demand on reality. The formalism itself doesn't demand this singularity; MWI does.

My argument is that a statistical and epistemic interpretation of the wavefunction—one that views it as a mathematical description of knowledge, probabilities, and ensembles, rather than an element of external reality—is a natural container for both quantum and classical dynamics. The distinction between them lies purely in how intrinsic uncertainties are related to the algebra of dynamic observables. It's why classically there's no Heisenberg uncertainty but both x and p have uncertainty relations with their conjugate Fourier representations nevertheless. That's all highly indicative that the way to think about all of these things is that the kind of dynamics you get out depends on the kind of statistics you want to see in your distributions. There is no non-statistical quantum effect. Not one. You cannot predict a quantum jump. That's fine - the physical why of that is not a question we need to answer, we just need to recognise what that does to statistics, expressed via a fundamental commutation relation. The algebraic formalism itself is scale-free and pure, and works beautifully. It's the interpretation of that formalism that must be intellectually honest about what it's describing across all scales, from the smallest quantum system to the largest classical one. It’s not about reinstating classical realism; it's about saying MWI is absurdly non-minimal in its ontological commitment, even classically. You’re welcome to your neo-Kantian phenomenal perspective, but if you want your external reality model (the universal wave function) to be epistemically responsible, it must avoid making the same commitment in a classical world where that commitment is readily dismissed.

EmeraldBeetle

SepiaBeetle If your interpretation (like MWI) claims the quantum wavefunction \Psi is an ontological object—the real, evolving state of external reality—and that this reality splits every time a quantum measurement occurs, then that same ontological claim must hold true when \hbar \to 0 and the system behaves classically.

AFAIU the classical limit for MWI, environmental decoherence suppresses interference, making branching dynamically vacuous and giving an effectively single quasiclassical history, with commutators tending to zero. That’s exactly the point though, in that one of the criticisms of MWI is that it just reproduces classical outcomes in that limit, so you can happily apply a classical, commutative algebra. So where’s the absurdity if the “branches” have effectively 'disappeared'?

Plus, while of course there are “no non‑statistical quantum effects”, those statistics aren’t classical but rather quantum (as in non‑Kolmogorov). And I still think KvN (with its commutative algebra and classical stochastics) runs into Bell/contextuality constraints that your McCaul et al. paper doesn’t appear to even discuss.

So I think KvN obviously works nicely as a reformulation of classical mechanics and looks like it's continually being refined, but (unlike McCaul et al.) you’re overstating its ‘deflation’ of quantum nonclassicality and especially its ‘critical’ relation to MWI. But that’s where the theoretical debate has been stuck for quite some time now, which is why I passed over KvN a while ago on my way to 'mad‑dog' Everettianism 🙂

I'm happy to have Everett and his nonclassical friends proved wrong but will have to wait on the next round of KvN rebuttals (I find Wallace quite readable) to see whether your classical nostalgia for phase space realism has finally won the quantum foundations day. Care to wager?

RustSilverfish

SepiaBeetle I had been drawn to this essay by its (frankly brilliant) title and abstract a couple of times, but must admit that in my first few attempts at reading it through I always bounced off of your 'note on style', which reads to me like a preemptive dulling of your blade lest it cut too deeply. If you want to go in, guns blazing, taking no prisoners, then do it! Don't hedge your bets, don't try to prime your audience into preemptively excusing any stylistic overreaches. If you want to draw blood, you should write to the courage of your own convictions, not flinch away from them at the last second!

Nevertheless, I am in broad sympathy with at least your cautions against exaggerated claims that are good for grabbing headlines and funding, but perhaps don't quite deliver on their promise in the long term. It is always good to erect a bit of a dam against the torrent of hype, and one can't deny that slapping 'quantum' in front of virtually any field of study invites a magnitude of hype hard to explain with classical resources. Hence, I do broadly agree with your call to invite quantum biology to distinguish itself as a distinct field of study, rather than just a distinct means of attracting funding.

Additionally, the argument towards an algebraic formulation of biology is interesting, but unfortunately cut too short to be convincing -- perhaps the essay would have benefitted from less airing of the familiar grievances regarding the publish-or-perish culture of academia in favor of more exploration of this intriguing idea. As it is, it feels somewhat tacked on, more of a teaser than an argument, which ends the essay on a somewhat incomplete note.

Let me now first get a couple of -- ultimately minor -- quibbles out of the way. First, the 'randomness' of quantum theory is, to my mind, far from its most distinguishing feature. After all, it can consistently be formulated in a perfectly deterministic fashion, in either the guise of Many Worlds or Bohmian theories. Furthermore, even intrinsically stochastic theories are by no means necessarily quantum. Also, I'm not sure about the claim that non-commutativity fully explains quantum weirdness. Again there are also theories with non-commutative measurements (Spekkens' toy theory, for one) that fail to reproduce certain features of QM (Bell-nonlocality and contextuality, for example). In the reconstruction of quantum mechanics of Clifton, Bub, and Halvorson, non-commutativity follows from the no-broadcasting axiom, but it needs a further one making unconditionally secure bit commitment impossible to yield entanglement, which, as you say, is 'quintessentially quantum'.

On another minor note, you write that 'The essential problem for almost all scientific inquiry is to find the correct representation'. I don't think this is the case at all. Much progress is made, in fact, by changing representations: changing Hilbert space operators for path integrals or a noncommutative phase space algebra. I don't even think there's a good case that there is a 'right' representation: but different ones may work to our advantage in different settings.

As for your overarching argument, I must confess to always having trouble with such 'a priori' declarations of impossibility. There are many things well-known from theoretical considerations that just ain't so. There clearly is space for quantum effects to be efficacious in the macro-regime: that's after all what people are doing with Bell test experiments and the like. An argument that such effects are irrelevant for biology then can't come solely from the physical side, but needs to take into account the biological contribution, as well: effectively rendering it impossible for biological systems to reach whatever narrow sliver of quantum effectiveness can be ported to the macroscale. Is there a gap between what quantum physics provides and what biology can reach, or do both overlap? I don't think the science on either side of this is advanced enough to settle, and moreover it seems to me that even answering it in the negative, requiring expertise both in biology and quantum theory, could well be called an exercise in quantum biology.

Anyway, I thought this was an intriguing essay and, despite my initial stumbling, well worth reading. I hope it'll do well in the contest!

SepiaBeetle

RustSilverfish

Ah, once again I've missed the boat on responding to things in a timely fashion! Still, I thought I'd register my appreciation for your thoughtful message, and try and compose some broad responses. I found reading it that the points you raise sit in the productive zone of being things I half agree with.

On the first point - I take that statement as the tax on polemical tactics. That whatever you do, you should lay your cards on the table. Wounded pride tends to generate hostile readings, so why not bare my neck, admit the vanishing possibility I might be mistaken, and invite the fight? That rightly or wrongly I'm writing to my own standards. You would be amazed how far that gets you in peer review, because it allows you to clearly discriminate between substantiative and stylistic objections, and simply ignore the latter. And in the end the only things I wanted to say but didn't were cut purely in the interests of space.
Moving on to your point about algebraic formulations of biology - I agree, it is interesting! But it's unfortunately also something where I simply didn't believe it was possible to make that argument simultaneously simple, persuasive, and concise. Certainly not without more mathematics. Moreover, in this instance attempting to sketch the idea has spawned a small project to formalise it. So eventually there'll be a paper on it, I'm sure! All this is to say that I don't disagree with your comment, it was just one of those cases where the constraints of the format forced a less than ideal compromise. On the other hand the sheer hackery of academia will always be with us, and it is very hard not to look at this field and see it overflowing with the pathologies of institutional sclerosis. And this shallow, madlibs approach to hard problems is . Just because the critique is familiar, doesn't mean it's not true. I simply do not understand how any principled person can look at the state of research, and not feel mortal outrage at it. Friends often call me a Jeremiah though, so perhaps this is merely my dogmatic character seeking release.

Beyond this, I'm afraid I will have to take up the gauntlet laid down by your quibbles. Since you suggested I write with the courage of my convictions, I shall put it bluntly. I think the vast majority of foundational and interpretational work is worse than useless, and should be treated with genuine contempt. It is playing at physics without a referee, a field runs out of experimental gradient but keeps the full institutional machinery of “serious physics” running anyway. You still have journals, tenure tracks, prestige hierarchies, conferences – but almost no way to falsify anything. So people start doing theatre: all the external trappings of hard science (maths, formalism, the word “theorem”) without the one thing that actually disciplines it: contact with the world.

For example, let us take the point of randomness. Yes, one can choose to interpret the unitary evolution of Hilbert space vectors in any manner of whacky ways, but as far as I can see all that equips one with is an endless argument which can never be wrong. The only observable in foundations is status. If we bite on the hard reality, what we see is that the outcomes we actually observe are genuinely incompressible. Regardless of what is producing them, any local measurement (and whatever anyone may claim, we have nothing but local measurements, simply existing at different levels of description) is random.

You are of course right that one can have a classically random formalism too! But the point is that without a commutation relation this is always a tunable feature. This relates to your other point. Very simply, you can show formally that if we take the limit of commuting observables we recover the boring old Liouville equation for classical dynamics. You can put a delta function initial condition into that and the randomness is simply gone. I'm sure I've mentioned it a number of times, but I'll do so again because I think it's a result that is actually devastating to these exotic interpretations when properly understood ( https://journals.aps.org/pra/abstract/10.1103/PhysRevA.108.052208). It doesn't require any radical assumptions, you simply parametrise the degree of commutation, and in the right representation the generator of your dynamics becomes manifestly classical as the commutation relation is taken to zero. So we can talk about states and correlations until the cows come home, but it's dynamics that distinguishes quantum and classical, and the rest is simply formal dressing up. I can find precisely the same 'quantum' behaviours in the classical theory, if I extend it it to consider the action not just of my phase space observables, but their conjugate Bopp operators. And the latter might not be observable, but they are coupled to the dynamics of the physical operators, so they are in principle inferrable from the observable dynamics! So like it or not, your interpretation has to explain the classical world in precisely the same terms. That's no problem if you do the sensible thing of simply accepting what the formalism and experiments are telling you. That this is a statistical theory. I have to say one of my favourite essays on this topic came as the winner of this contest a few years back: https://qspace.fqxi.org/competitions/entry/2073. Well worth a read I think. Now having said all this, people can obviously believe what they like - and these questions are certainly more of religion than of science - but widening your formal lens even slightly makes most these foundational programs look manifestly silly, at least to me. It looks more like highly-trained people defending narrative territory over an empirically frozen core, than any particularly deep insight into the fundaments of reality.

Your point about representation confused me slightly, because your example seems to back up precisely the point I am making. Problems that are intractable in one representation become trivial in another. So solving a particular problem is almost entirely a case of finding the 'right' representation for it. Actually the waveoperator paper I linked above is an excellent example of it, because it finds the representation where these otherwise singular classical limits become well behaved. That's not trivial, and it explains quite nicely where so much of this pathological behaviour in semiclassics is really coming from - limit of a trace is not at all the same as trace of a limit!

So, on your last point, I can boil it down to a very simple question. Do you trust the formalism of quantum dynamics? I think anyone who makes these arguments needs to be sat down and forced to take three courses of condensed matter physics, and see if they feel the same way. Because my argument fundamentally isn't just a priori, it's one of the main lessons of by far and away the largest and most exhaustively researched area of physics. It's where the actual theory lives and dies, and has to earn its keep by making actual predictions. If you're only in contact with the gnomic proclamations of deep thinkers, I think it's easy to be left with entirely the wrong impression of what quantum theory does and does not say. Check out this genuinely immortal piece (whose title I stole): https://www.science.org/doi/10.1126/science.177.4047.393 - I think it's a fairly persuasive argument for the inherent absurdity of quantum biology.

Okay, that's it! Thanks again for the message. I had fun reading it, and had fun spinning up this rant

« Previous Page