Though I agree that 'physics and mathematics have a common basis', there is a fundamental difference. If in a self-creating universe the sum of everything inside, including spacetime itself somehow must stay zero (conservation law), then the physicist cannot treat physical quantities and phenomena like a mathematician does his numbers and symbols. Where his numbers and symbols exist only in the mind of the mathematician, the physicist is part of the sum he studies, so he is not allowed, for example, to imagine how the universe looks from the outside as any physical observation requires a at least a specification of his position with respect to what he observes, his distance and the conditions at the place he looks from, which by definition is impossible outside the universe. If he nevertheless imagines to see galaxies of about the same age (since he believes their particles to be created at the same time) receding from each other faster than light, then this is only possible by seeing their spacedistance, but ignoring their corresponding timedistance: by thinking like a mathematician.
'... The mathematical universe is safe from Godel's theorem ...'
I'm afraid this is not the case. If to explain some phenomenon or prove some theorem we start our reasoning from assumptions and axioms which contain preconceptions, if the truth of our allegations depends on the truth of unprovable assumptions and axioms, then we can never prove them in an absolute sense, however valid they may be within the set of axioms and rules of reasoning they are formulated. The problem is that though our assumptions and axioms may seem self-evident, they aren't necessarily true as they only reflect our view of our world and express a logic which may differ from nature's logic. Richard Powers (in The Gold Bug Variations) 'Why is the last line of a proof surprising, if its truth is already hiding tautologically in the lines above?' suggests as much: that we put as much information in our choice and formulation of axioms and rules of reasoning as we can get out of them. If the proof of a theorem to some extent also involves the proof of the implicit assumptions which are built into our axioms and rules of reasoning, then the formulation of a theorem can be thought of as an effort to formulate this implicit information explicitly, its proof being incorporated in the theorem as it is formulated. If in that case we don't so much prove something but rather adapt our thinking to the way our observation evolves, then the impossibility to (dis)prove statements which can be made within a consistent set of rules and axioms (Gödel) might originate in the incompleteness or indefiniteness of our definitions and axioms, in the lack of information or restrictions we've put into our rules, axioms and assumptions, so statements can inherently be too ambiguous to prove or disprove. The problem is that much of the information we put in them appears too obvious for us to consider as being information, as if it reflects a truth that needs no inspection: as it is almost impossible to be aware of this implicit information, we indeed are surprised at the last line of the proof, as if we got some information for free that we didn't put in ourselves in the first place. As our reasoning and the tools we think with are rather the product, the expression of our relation to our world than something which is open to inspection (by itself), it is difficult to detect the implicit information present in our assumptions, in the preconceptions they may contain. This might mean that if we could explicitely formulate all implicit information in a set of axioms and rules so there would be no ambiguity, nor in the theorems we can formulate within that set, Gödel's theorem would no longer apply, any statement or theorem being a tautology. If we have more confidence in a theory as it is more consistent and it is more consistent as it relates more phenomena, makes more facts explain each other and needs less additional axioms, less more or less arbitrary assumptions, then any good theory has a tautological character though a tautological theory of course isn't necessarily true nor useful.
In an uncaused, causeless universe which creates itself (see Mechanics of a Self-Creating Universe), where things and events create each other, they explain each other in a circular way, are each other's 'cause'. Though a circular reasoning at first sight may seem ridiculous, here we can take any statement, any link of the chain of reasoning without proof, use it to explain the next link and so on, to follow the circle back to the statement we started with, which this time is explained, proved by the foregoing reasoning. Though in a self-creating, noncausal universe a proof seems to be less convincing than a proof which follows a causal reasoning, a causal assertion or explanation ultimately is invalidated as the primordeal cause it is built upon by definition cannot be understood nor proved. The point is that if our logic originates in nature's logic and not the other way around, that our logic is but a reflection of our relation to our world and not a reflection of some absolute, platonic kind of truth which precedes, exists outside that world, an objective reality as there's no such thing, mathematics and its development follow physics, and not the other way around, so we cannot blindly rely on its conclusions that explain the why and how of our universe, its laws. Though dreaming up mathematics without bothering too much about the nature of the quantities its equations refer to sometimes can help decide whether ideas in physics make sense, mathematics itself cannot dream up really new physical approaches or ideas. An excessive emphasis on mathematics tends to create its own reality and confuse our view on physical issues. Though many models in physics may mathematically be consistent, I'm still waiting for the one model which obviously, compellingly and necessarily excludes any other model and explains why the universe needs the particular particles we find, why the ratio between their masses is as it is etcetera.