Dear Matt,
We have enjoyed reading your essay and broadly agree with you. Fully agree about the dynamic tension between physical theories and mathematics...what we call `frailty of the connection'.
A few remarks on the middle ground between quantum and classical mechanics, in the light of your conversation above with David Garfinkle. We fully agree there is a big difference between
Newtonian gravity - PPN - General Relativity
on the one hand, and
Classical Mechanics - ?? - Quantum Mechanics
on the other. With hindsight, we feel the reason for this difference is apparent. GR, as we agree, has a built in structure which allows it to reduce to Newton's gravity for weak fields and small speeds. As is well-known, there is no analogous built in structure in quantum theory which permits the recovery of classical mechanics in the limit. We believe the biggest difference between quantum and classical mechanics is the absence of macroscopic position superpositions in the latter theory [this of course being the root of the measurement problem]. And quantum mechanics is unable to explain this, because it claims that even for large masses position superpositions must be seen (Schrodinger's cat). To us this is an indicator that quantum theory is incomplete and approximate.
It is tempting to believe that the theory of Continuous Spontaneous Localization [CSL] which is the continuum version of the GRW you allude to in your essay, is a worthy phenomenological candidate for such a middle ground, taking a role somewhat analogous to PPN. There is a new constant of nature in the theory, the so-called rate constant, which is proportional to the mass of the object: it goes to an extremely small value for small masses, so that CSL reduces to quantum mechanics in this limit. For large masses, the rate constant is large enough that CSL (being a stochastic nonlinear theory) destroys macroscopic positions at a rapid rate, causing wave-function collapse, and hence explaining the measurement problem and the Born probability rule, as well as recovering classical mechanics. Thus CSL seems to provide a nice universal dynamics, with the quantum and classical as limiting cases.
We agree with you that CSL has its own limitations [it will be interesting to know though, which limitations you regard as serious ones]. Nonetheless, the fact that CSL makes experimental predictions in the middle ground which are different from the predictions of quantum mechanics, and that these departures are testable in the laboratory and are being tested, makes it an attractive benchmark against which we may evaluate quantum mechanics.
With best regards,
Anshu, Tejinder