Dear Ben,
Quantum Gravity?
Their of course does seem to be a distinct differentiation in the dynamics of interactions at quantum scales. As you discuss in your essay, foundational theories are evaluated by their explanatory and predictive success. While many think that general relativity's primary advantages over classical gravitation is its predictive accuracy and it's ability to explain HOW gravity works. I can't assess, but it seems to me that much of GR's success in predicting the orbit of Mercury involves a proximity 'fudge factor' of its own. I may have misunderstood...
However, when if comes to explaining how gravity works I think GR is only an incremental improvement over Newton's metaphysical 'attractive' force. As a systems analyst, I can only view GR gravity as a system of abstract dimensional coordinates that very accurately describe the EFFECTs of gravity, not its physical causation. It seems to me that there is no description of any physical aspect of dimensional spacetime that is represented by the described coordinates.
Meanwhile, as I understand the 'vacuum' of dimensional spacetime seems to always contain some kinetic energy that, in flux, is sometimes manifested as 'virtual particles' that do not comply with the universal proportions of matter and and antimatter. It seems that whatever condition(s) that produced the preponderance of matter in the universe is no longer effective. That aside, I think the annihilation of matter-antimatter particles in the vacuum of space is a direct measure of its kinetic energy content. Back to this in a moment...
Its often discussed (like a classroom trick) that gravity is much weaker than the other forces of matter. One demonstration of this is using a small magnet to pick up a paper clip, overwhelming the gravitational 'attraction' of the entire Earth! No mention of that small magnet attracting the Moon, however. My point is that gravity IS fundamentally different that the 'other' forces of matter - IMO it's not a force of matter at all!
It seems to me that the initial or original expansion of universal spacetime must have infused spacetime with all the energy that was not converted into matter. I speculate that gravitational effects are produced by boundary interactions between this kinetic energy of spacetime and the localized potential mass-energy. While all the accelerating/compressing effects of gravity are the direct result of gradient fields of kinetic vacuum energy contracted by localizing potential mass-energy. BTW, Newton's 'attractive force' would be an approximation of the interaction between two opposingly directed gradient fields of vacuum energy.
I think these ideas could be tested by measuring the rate of virtual particle-antiparticle annihilations within a vacuum chamber on Earth, and in orbit or on the Moon. I have no idea of the magnitudes involved (the annihilation rate would have to be av very indirect measure of vacuum energy density), but if gravitation involves the kinetic vacuum energy of space, represented by GR's curved spacetime, there should be some measurable differences within a gradient gravitational field.
I also have some wild ideas about the physical nature of particle mass as the reconfiguration and redirection of wave propagation energy producing the energy absorbing property of inertia, and the particle property of mass as the particle-wave manifestation frequency. But I'm starting to run out of steam.
You'd have to be highly imaginative to have much understanding of what I'm attempting to describe in this last bit, since I don't really know the proper terms, or at least I'm not conversant. BTW, I have mentioned these more fanciful integrated system design ideas to a couple of other authors who have expressed some interest, but we'll have to see. I only briefly touched on these ideas here, but would be more than happy to discuss further if you happen to be interested. The truth is that, if these silly thoughts were by some chance the keys to unlocking the secrets of the universe, my hands are tied, I'm too short and I won't be around long enough to use them. They're free for anyone else to use...
Please do seriously consider my much more grounded interpretation of dark matter, at least. I think it's only a 'fudge factor'. I'm also very skeptical of the accelerating universe hypothesis, but its analytical requirements are not nearly as simple as the silly 'dark matter' incident. I like to consider that the observational evidence centers on the oldest available type Ia SNe observations, those that also represent the prevailing conditions of the earlier universe. But then the discrepancy with standard cosmological models used to estimate distance from redshift are also involved.
I apologize for rambling & appreciate any consideration you might give. Please let me know if any clarifications would be useful.
Sincerely, Jim