Garrison Keillor has his "Guy Noir," who "On the tenth floor of the Atlas building still seeks answers to life's persistent questions." If you have ever listened to his "Prairie Home Companion" you know this well. There are persistent questions, such as "Does God exist," that will probably never be conclusively answered.
The problem is that we transition from physics to metaphysics when we start pondering what the relationship is between mathematics and physics. Smolin has written an essay that I think rather favorably of, but his stance is naturalism, which is not something one can prove. Naturalism is a conjecture about things that is really metaphysics. Platonism, which in some ways is a bit too mystical for my tastes, is also a metaphysics.
What I outline is a mathematics that has what I tend to think of as meat or body. The pure mathematics taught and studied in mathematics departments often involves what might be called "soul." I am not out to disprove or even disavow this soul, but I do tend to think that it has a questionable applicability to modern physics. This is particularly the case with matters of infinity or the continuum. Mathematicians are mostly objectivists who consider their work to involve the discovery of "something," which is not physical. This is an appeal to the existence of this soul, or in its extreme form a Platonic reality. That is fine with me, and I may put on the cap of Platonism when it suits me, and take it off at other times. In my essay I tend to keep the cap off. My essay is largely concerned with what sort of practical aspects of mathematics are likely to impact physics in the next few decades.
I think these more metaphysical questions are not going to be answered, or answered very easily. There is Tegmark's conjecture of the MUH (mathematical universe hypothesis), where in view of Goedel's theorem he replace M with C for computation. This may be the case in some ultimate sense. However, I don't see how this can ever be empirically supported. This may amount to trying to "prove too much." This attempt to get away from dualism between mathematics and physics is a sort of monism. The debate between monism and dualism may never be resolved. The two are sort of the two sides of a Buddhist satori.
LC