Eric,
Firstly, congrats on a well-worded essay highlighting the often under-estimated role of geometry in Physics.
To that end I would like to offer you a solution that provides just such an answer that you hint at in your essay - namely my theory Tetryonics - the charged geometry of EM mass-ENERGY-Matter [as outlined in my essay of the same title].
Its priori foundation is that Energy has a equilateral [triangular] geometry and from that geometry all mass-Matter geometries arise along with our Forces and Constants etc. You will find a great deal more information on these processes on my YouTube channel [extending the outline offered in my essay].
I agree totally with you that it is the geometry that defines our mathematical formulations and without a geometrically defined model on which to base our Math we constantly need to refine it in order to better reflect observed physical phenomena.
Two illustrations are attached to that end highlighting geometric solutions to Bell's formulation and wave-functions [as they apply to the equilateral geometries of Charge and EM waves respectively]
Of note is the fact that the current Maths does not change [except where it is incorrectly formulated] only the underlying geometry changes [equilateral vs spherical]
To date I have applied Tetryonic geometry to QM, QED, Chemistry, Cosmology, SR & GR with outstanding results and I trust you will find then enlightening as well.
I know you will find common ground with Tetryonics and the principals you have outlined in your essayAttachment #1: Figure_24.04__Heisenberg_Uncertainty_Principle_800x600.jpgAttachment #2: 1_Figure_39.07__Matter_in_motion_800x600.jpg