Essay Abstract

Quantum theory is extremely successful in explaining most physical phenomena, and is not contradicted by any experiment. Yet, the theory has many puzzling features : the occurrence of probabilities, the unclear distinction between the microscopic and the macroscopic, the unexplained absence of superpositions in positions of macroscopic objects, the dependence of the theory on an external classical time, and the experimentally verified but peculiar `influence' outside the light-cone in EPR experiments. These puzzles point towards a conflict between quantum theory and our present understanding of spacetime structure, and suggest the existence of a deeper theory. In this essay we make the case that in the underlying theory the matter and spacetime degrees of freedom are non-commuting matrices, and yet the dynamics is analog. A digital quantum-theory like dynamics for matter as well as spacetime emerges in the statistical thermodynamic approximation to this deeper theory. When most of the matter clumps into macroscopic structures, it is shown to behave classically, and it induces classical dynamics on spacetime; this is the eventual analog limit, our macroscopic world. In between the digital layer and the uppermost analog layer is the realm of standard quantum theory - microscopic objects and their interaction with measuring apparatuses on a classical spacetime background : the semi-digital approximation. Such a multi-layered description of dynamics can explain the puzzling features of quantum theory, and is testable by ongoing laboratory experiments.

Author Bio

Professor of Physics at the Tata Institute of Fundamental Research, Mumbai, India. Research Interests : Quantum Gravity, The Quantum Measurement Problem, The Origin of Dark Matter and Dark Energy, Formation of Large Scale structures in the Universe, Gravitational Collapse of Compact Objects and the Cosmic Censorship Hypothesis. Member of FQXi and Recipient of the John Templeton Foundation Grant (2011) for research on Quantum Measurement. Fourth Prize in FQXi essay competition 2009. Three time Gravity Research Foundation Essay Prize Awardee. Other interests : Teaching and Science Popularization. URL : www.tifr.res.in/~tpsingh

Download Essay PDF File

  • [deleted]

Tejinder,

I think a basic way to peel time out of the equation is to consider that we may be looking at it backwards. We experience it as a flow from past events to future ones, but the physical reality is that it is the changing configuration of what is that turns future potential into past circumstance. We don't travel from past to future, the future becomes the past because the configuration changes. Tomorrow becomes yesterday because the earth rotates.

This way, time is an effect of motion, similar to temperature, not an external factor which must be considered.

There can be no dimensionless point in time, as that would freeze the very motion creating it, much like trying to take a picture with the shutter speed set at zero. As with a temperature of absolute zero, it would freeze motion. Thus objects, macro as well as micro, cannot be truly separated from their motion.

So when we try measuring anything, it becomes a balance between precision and contextual inclusion. Much like taking a picture is a balance between clarity and depth.

Time is a serial measurement, while temperature is a scalar measurement. These two balance each other, just as the two sides of the brain balance each other. The left, serial processor side is essentially a clock, in that it records the cause and effect of action. While the right, parallel processor side is a thermostat, in that it measures the non-linear energies. As a physical entity, our brain proceeds from past events to future ones, while our mind, as the recording of these configuration states, is the memory of these events being created and receding into the past.

The fact though, is that it is what is physically real that is the constant and it is the configurations which are emergent, that's why it makes more sense to view the arrow of time as the future becoming the past. In this way, multiworlds isn't necessary, as it's this very collapse of probabilities which creates time. Yes, for a microsecond, the cat is both dead and alive, because the point of death is not a dimensionless point. In fact, for some, it can take quite awhile. The wave only collapses from the subjective perspective, as its energy continues in other forms.

    • [deleted]

    Dear Sir,

    We have a wrong notion about time traveling backwards. We have discussed this aspect in our essay. You are welcome to comment on it.

    Regards.

    basudeba

    5 days later
    • [deleted]

    Dr. Singh: I am glad to see that you have discussed decoherence in your essay. Dieter Zeh has made a convincing argument that all "particle" or "quantum jump" phenomena are in fact continuous in the light of environmental decoherence. I wonder what you might think of the discussion of decoherence in my essay. I describe how decoherence apparently only happens in an interaction between subsystems, such as an object and its environment; that closed, unpartitioned systems do not experience decoherence; but that the nature of this partitioning in the early universe is unclear, leading to a cosmological expression of the measurement problem. I propose a couple of models that may resolve this problem. I am curious why this question doesn't receive more attention, given that Penrose and others have pointed it out. Best of luck....

    Dear Professor Tejinder,

    I enjoyed reading the first part of your paper where you expressed succinctly and clearly your discomfiture concerning the fundamentals of Quantum Mechanics. I regret that the second part was too mathematically abstract for my understanding and interest.

    You noted "Moreover, unlike in the case of tossing of the dice, where the probability of any one outcome can be reasoned to be 1/6, quantum theory provides no explanation for the Born rule, simply adopting it as an ad hoc postulate which agrees well with experiment."

    A very specific suggestion for the physical basis of quantum probability is contained in section 2.7 (particularly Figs. 28 and 29) of my original 2005 Beautiful Universe paper on which my present fqxi paper is based. There I propose that the basis of quantum probability (and many other phenomena such as radiation in the vacuume) is in the diffusion of angular momentum in units of (h) through a crystal-like universal ether lattice of nodes. Your expert feedback is welcome on this and on any other aspects of the theory..

    Kind regards, Vladimir.

    Hi Tejinder, you seem to talk a lot of sense. I have a nagging question: why can't the Archimedes screw be the answer to solving the particle/wave duality paradox? I think it does. Am I wrong?

      Dear Alan,

      Thank you for your kind remarks on my essay.

      Could you explain how the Archimedes screw explains wave-particle duality? Then we can surely discuss it further.

      I wish to take this opportunity to mention, as pointed out in my essay, that in my picture wave-particle duality is not a paradox. The basic nature of fundamental entities in quantum theory is wavelike and described by the wave-function. Only when this wave interacts with a macroscopic apparatus and gets localized [via wave-function collapse] do we attribute it a particle-like characteristic.

      Best regards,

      Tejinder

      • [deleted]

      Not only can it explain particle/wave duality but also gravity and dark energy, believe it or not. It's the mechanical nature of the helical screw which is important. The idea of the fundamental nature of reality having structure. It's this structure in a helical configuration which can cause a force of attraction if this dynamic particle interacts with another particle. If this Archimedes screw then travelled around a hypersphere, or wraparound universe, then it would emerge on the other side as a force of repulsion i.e. dark energy.

      You have to physically plat with a wood screw or a bottle opener to really understand the simplicity of this idea. Many thanks for taking the time if try this. Alan

      (P.S I forgot to login the first time)

      Also Tejinder, if you can lay your hands on two cork crew bottle openers, then you'll see that they can travel past on another in a helical ring-donut configuration. Two spinning helical threads could exist in a double-helix configuration, either travelling in opposite directions or the same direction i.e. like neutrons and protons/anti-protons? Do you begin to see what I'm getting at?

      Dear Tejinder,

      You write

      ---"the puzzling features of quantum theory"---

      If a universe is to create itself out of nothing so particles have to create themselves, each other, then they are as much the source as the product of their interactions, the cause as well as the effect of each other's existence. If so, then there's nothing puzzling about quantum mechanics. For details see my essay.

      Kind regards, Anton

      I have been reading with pleasure your highly technical Essay and I think that may be useful to highlight some analogies and differences between our respective approaches.

      As explained in my Essay, the canonical theory goes beyond less fundamental approaches as the superstring theory. This includes also the matrix formulation of M-theory (based in a Trace dynamics as that summarized in your Essay). The general state \hat{rho} of any system in canonical theory is defined on a generalized Liouville space (see pag 6 in my Essay). We can use an extension of Wigner techniques to derive an equivalent Wigner state \hat{rho}_W, but defined on a non-commutative phase space ({x},{p}), with associated N x N matrices like in Trace dynamics. However, the information contained in this non-commutative phase space is redundant and whereas the simplest dynamics (e.g. the S part of the canonical dynamics) can be adequately described by the geometrical star-products, there are difficulties to obtain the more general dynamics.

      Indeed, the generator of the time translation in the canonical theory is not given by an operator but by a superoperator (e.g. the coefficients Omega are represented by superoperators in the quantum formalism associated to the post-Keizer form obtained at our Center). Those superoperators are defined in a superoperator space and are associated to (super)matrices of order N^2 x N^2. The evolution is described by an operator (matrix) Hamiltonian only as approximation in the 'pure' limit (see Box 1 in reference 8 in our Essay). From the Hamiltonian, we can obtain Lagrangians and actions using the usual techniques. That is, neither Hamiltonians nor Lagrangians and actions can describe the dynamics of the processes in the general case. Also the usual bosonic and fermionic commutators/anticommutators are valid only in the same 'pure' approximation.

      Next, you introduce a statistical mechanics by coarse graining, using a maximum entropy method. There is several difficulties with this approach, which are solved in a natural way in the canonical theory. The first is that the canonical theory belongs to the modern statistical mechanics School of fine grained methods [FG], eliminating the extradynamical coarse graining considerations and their associated paradoxes. The second, is that the assumptions as your "the ensemble does not prefer any one state in the Hilbert space over the other" are derived rather than merely postulated. The third is that these assumptions work for equilibrium but not for far from equilibrium systems; therefore, we can obtain a nonequilibrium statistical mechanics of broad scope, whereas the work of Adler and others (reference 1 in your Essay) is limited to the simple case of (stable) equilibrium. The fourth, is that your work and the work of Adler and others relies on the Liouville theorem, which means that the resulting equations of motion cannot describe the evolution of the non-conserved variables, doing impossible the link with the phenomenological theory of nonlinear nonequilibrium thermodynamics. This is a well-known deficiency of the older approaches to statistical mechanics, which is solved by modern formulations of nonequilibrium statistical mechanics as that by Byung Chan Eu [Eu]. We can derive his nonequilibrium statistical mechanics as an approximation to the more deeper canonical theory.

      As you correctly point, the next step in the scheme is to consider fluctuations. Again, at this point the work of Adler is based in further approximations. Effectively, he obtains a stochastic generalization of the Schrödinger equation and this generalization allows us to describe the nonlinear phenomena cannot be described by the usual Schrödinger equation, such as the collapse of wavefunctions. His work and that of others is very fascinating, because instead of the dual structure associated to the Copenhagen formulation of quantum mechanics, we have a single evolution, which gives reduction for measurements and the usual unitary evolution when the system is not measured! However, this kind of work is not fundamental. The canonical theory includes fluctuations (see Box 1 in reference 8 in our Essay) but cover a more broad spectrum. It is only when we approximate the canonical rate theory by its weak-coupling limit and take the Markovian limit that the fluctuations are Brownian. Moreover, Brownian fluctuations are related to what Nico van Kampen named "extrinsic noise", which is due to the fact that the system (the Brownian 'particle') is open. However, there is also another kind of noise, "intrinsic noise", which is related to the fact that the underlying structure of matter is discrete rather than analog. As van Kampen correctly noted, the noise observed in chemical reactions is of the intrinsic kind. It is not surprising that the canonical theory is able to describe both noises.

      Then you go over discussing the role of time and you introduce two concepts of time; at the one hand, the concept of dimensional time, which, as you correctly point out, is associated to a quantum operator of time and, at the other hand, the concept of "affine parameter". This is not very different from the Stuckelberg extension of string and p-brane theory (see reference 15 in my Essay). At this point, your essay looks ambiguous for me and you refer to a reference still in preparation. You say that you introduce operators (q,t) for each particle. However, next you write that this "introduce a non-commutative Minkowski spacetime". I assume that you mean a 4N dimensional non-commutative spacetime (for N particles), which would reduces to a non-commutative Minkowski spacetime only in the one-particle case.

      In the Stuckelberg extension of string and p-brane theory, the introduction of a phase space as your (q,t,p,E) yields dynamical redundancies. These are 'solved' (or at least alleviated) by working off-shell. What approach do you take in your reference 11 in preparation?

      Another point that I want to comment is when you take a Block Universe view where your "affine parameter" tau is not identified with time. Here your work departs from the Stuckelberg extension of string and p-brane theory (see reference 15 in my Essay). You seem to support what Pavsic names the interpretation "(i)", whereas he prefers the "(ii)". You give none technical reason for your interpretation, whereas others (mainly Horwitz, Fanchi, and others) have given many details on why the interpretation (ii) is the correct. In the references 17 and 18 cited in my Essay I showed that the parameter tau in the Stuckleberg, Horwitz, & Piron theory reduces to Newtonian time for interacting charged particles and for massive particles under gravity respectively, whereas the concept of dimensional time associated to spacetime rigorously fails to reduce in both cases.

      You argue that the "transition from the lower analog layer to the upper analog layer also helps understand the

      origin of the arrow of time". However, you give no technical detail and merely state that the question of the origin of the arrow of time is related to "why is the initial entropy of the Universe so low?". This is the well-known cosmological argument, but unfortunately it fails when one considers the details. A better approach to understand the origin of the arrow of time is given by the Brussels-Austin School [IRREV]. It seems that the Brussels-Austin School last theory for LPS with Poincaré resonances can be obtained from the canonical theory for systems with bifurcations [IRREV].

      [FG] The quest for the ultimate theory of time, an introduction to irreversibility

      [Eu] Nonequilibrium Statistical Mechanics, Ensemble Method 1998: Kluwer Academic Publishers, Dordrecht. Eu, Byung Chan.

      [IRREV] Trajectory branching in Liouville space as the source of irreversibility

        • [deleted]

        Dear Tejinder,

        I wanted to let you know that I have read through your essay. I have not come across this kind of multilevel consideration of reality before and so it catch my attention. Although not having a mathematical background I did also find it hard to follow. The lack of beaks in the text also made it visually daunting. It is the kind of work that I would need to tackle slowly and a little at a time. However I appreciate that this was not written for the likes of me but rather mathematically competent physicists.

        I have given a lot of thought to what is meant by reality and the question of time.You did point out an unknown underlying mechanism. That is something I am addressing in my essay. I am not sure if you were saying that the multilevel analysis was a potential solution or that it could not give full answers because of the unknown foundational mechanism. As you can probably tell I couldn't follow it very well, even though I think you had something very interesting to tell us. The fault lies with my lack of necessary mathematical education, abilities and skill.

        I do hope that you find lots of interested readers, who are competent in maths and can give you more constructive and positive feedback.

        Wishing you good luck. Georgina.

          Dear Juan,

          Here I try to answer the questions posed by you in your post above.

          The non-commutative Minkowski spacetime I talk of is defined in the technical endnotes. In this spacetime there live the `particles' [i.e. matrices], each having coordinates (q,t). I only have in mind a noncommutative generalization of ordinary mechanics, resulting here in a 4N dimensional noncommutative configuration space for N particles.

          Regarding the introduction of the phase space (q,p,t,E) : The resulting dynamics is more general than quantum dynamics, as I discuss in my essay. So it is not required that one works off shell. The reduction from here to ordinary quantum theory, where time is not an operator, is discussed in the essay : the dominant part of the Universe must become classical, providing a classical time, for us to be able to formulate quantum theory in the standard way.

          Regarding the Block universe argument : My reason for suggesting a Block Universe in the underlying analog dynamics is just that I do not see any other way out, when classical time has been raised to operator status. But I will see the references you mention, and try to understand what you refer to as Pavsic's interpretation (i).

          Arrow of time : I would not claim that the explanation I give is the only one or the best one. But I feel [subject to the cosmological argument being given : please let me know why you say it fails] my reasoning is plausible, even without trying to build up technical detail. If the Universe begins in a single microstate of the Block Universe described by analog dynamics, the Boltzmann entropy is zero by definition.

          I hope to make time to see the references on canonical theory in your essay. I appreciate your points that the canonical theory can deal with nonequilibrium situations, and with fluctuations in a manner more general than discussed in my essay.

          Once again, thank you very much for your careful reading of essay, and your detailed comments, especially regarding `Stueckelberg time'.

          Kind regards,

          Tejinder

          Dear Georgina,

          Thank you for your very kind remarks. I too have enjoyed reading your essay.

          I can try to summarize here the key points of my essay. I believe our present understanding of quantum theory is incomplete, because of the quantum measurement problem, and because of the presence of an external classical time in the theory. A *potential solution* is suggested by proposing that quantum theory is an approximation to a deeper theory. This deeper theory, motivated by the great work of Stephen Adler, is the classical mechanics of matrices [particle positions and spacetime coordinates are matrices]. this is an *analog* theory. The digital quantum theory that we know of emerges from this analog theory in the thermodynamic approximation. This emergence explains the puzzles of quantum theory.Furthermore one can one explain how our analog macroscopic world emerges from digital quantum theory. Thus one has a three layered description of dynamical laws : analog, digital, analog. The `three and a half' comes from noting that the world of our physics laboratories consists of quantum systems interacting with analog classical apparatuses in a classical spacetime.

          Good luck to you too, for the contest.

          Kind regards,

          Tejinder

          • [deleted]

          Dear Tejinder,

          Thank you for taking the time to summarize the core of the essay. It is helpful.I do agree with the first paragraph of your summary. Also that QM is part of a deeper theory. I would have it as part of a greater theory, encompassing both QM and space-time. Time being the "problem child".

          I also see a 3 level reality. Foundational level unobservable objects, data transmission( One might say the raw file of photon data) and emergent space-time image reality. QM models the unobserved timeless objects. Unique input selected by type and position of detector allows space-time construction. Emergent as a temporally distorted artifact from the transmission delay of data between object and observer.I suppose that as a non physicist my explanations may appear naive. Though Einstein did say "Everything should be made as simple as possible, but not simpler".

          I have said I am not competent to comprehend the work that you discuss, let alone evaluate it. So please forgive me for not doing so. What a dull world if we all thought in the same way and had identical talents and interests. It is good that you have brought it here for others to see and consider.

          Once again, Good luck, Georgina.

          • [deleted]

          Dear Tejinder,

          While your approach is quite different from mine, and quantum theory is not my primary concern, I nonetheless appreciate your list of belonging problems.

          I am not sure whether a peculiar 'influence' outside the light-cone has really been experimentally verified in EPR experiments. You gave no reference for your sentence: "The 'action-at-a-distance' in an EPR type experiment, which Einstein called spooky, has been experimentally proven to exist [however, we know it cannot be used for signalling]." If you refer to Nimtz type claims, I strongly disagree. To my knowledge there is no correct evidence for action at a distance.

          I tried to explain why Planck's constant has nothing to do with the non-commutativity.

          What I am tempted to ironically call the Schulman length between micro and macroworld might have a simple explanation: Quantum theory is affected by erroneous interpretation after arbitrarily choosing a complex ansatz. You may trust in my competence in this case.

          What about the allegedly compelling accuracy of agreement between predicted and measured values, I see two question marks. First I recall Lighthill's theory of cochlea whose results were tweaked very close to reality, even too good as to be honest, because the passive model did not yet consider cochlear amplification. I also recall not yet understood discrepancies, in particular concerning a paper by Gompf et al. Secondly, I suspect there might be a qualitative mistake even in case of quantitative agreement. Before I abandon the causality conjecture I am ready to put any theory in question, even spacetime.

          Please do not take it amiss if I consider it justified to look for possible mistakes at the most basic level, and this layer is in my understanding the fundamentals of mathematics. Do you object?

          Regards,

          Eckard

            Dear Tejinder,

            Thanks by confirming my above assumption about the 4N dimensionality of the spacetime in your Essay.

            When classical dimensional time is raised to dimensional time operator, the Stuckelberg-like evolution time continues labeling the evolution of the generalized quantum states, without any need to take a Block universe viewpoint. Indeed, as showed in the references cited in my Essay, the picture based in an evolution time is more fundamental.

            The cosmological argument cannot explain the arrow of time because initial low entropy states lead to evolutions incompatible with the second law. In the Chapter 1 of the BAS reference that I gave to you in the forum of my Essay, the authors show how an initial low entropy state belongs to both semigroups Lambda^{} and Lambda^{-}, whereas only Lambda^{} is compatible with the second law and the observed phenomena. They then choose the correct semigroup by comparison with observation. It is this selection which explains the arrow of time. Their demonstration can be extended to initial pure states with zero entropy and also to more general classes of dynamical systems.

            • [deleted]

            Your essay was thought provoking and I gave it a high score. I do take some pause with the conclusion that quantum theory is purely a statistical result. Experiments with W and GHZ states illustrate Bell inequalities for a single experiment. I will have to read this again to see how this can be reconciled accordingly.

            Cheers LC

              Dear Lawrence,

              Thank you for your kind response to my essay. I too plan to read your essay soon.

              You raise an important question. Actually, here the statistical character of quantum theory is relevant only in so far as its derivation from the underlying dynamical theory of matrices is concerned. Once one has that derivation, the underlying theory could be `forgotten', in the sense that one is not examining dynamics at the level of precision of the underlying theory. The stochastic nonlinear Schrodinger equation thus derived does describe an individual system, as you rightly point out. The stochastic term drives the quantum system to a definite outcome during a measurement, but because it is a stochastic process, one cannot predict exactly which outcome it is. Only the probability of the outcome can be calculated, and this is proven to be equal to the Born probability. You might like to see the nice discussion in Chapter 6 of Adler's book [Ref. 1 in my essay].

              Good luck to you in the essay contest. Cheers ...Tejinder

              • [deleted]

              Dear Eckard,

              Thank you for your interesting remarks and your criticism is most welcome. I plan to read your essay soon and will leave my comments on your page.

              By `action at a distance' I only meant the experimental verification of EPR correlations and Bell's inequalities by the experiments of Aspect and others. On this aspect, my view of quantum theory is conservative - there is no superluminal signalling.

              More soon, and with best regards,

              Tejinder