These are the ideas I am struggling with so I don't have clear answers, but here is how my thinking goes. Imagine you were able to run some simulation of a small world on a very powerful computer, so that within that simulation there is an intelligent entity that can examine its world. The entity could observe the artificial laws of physics that you were simulating, but it could never see what programming language you used or the hardware details of the computer you were using. I think this last layer must be similar in a way. We can't see the details of the ensemble of mathematical possibilities from which the bottom layer is emergent. We don't get to see the mathematical symbols or the choice of axioms.
There are also examples of this from physics. Near a critical point in some system of statistical physics you can get scaling behavior. If you take the limit at the critical point and rescale than the details of the statistical physics system shrinks to zero and vanish. A quick google search brings up this talk on the subject.
Both these examples are based around types of universality. In the first case it is the universality of computer languages that gives a definition of computability that is language independent. In the second case it is universality in critical systems. I think that something like this is happening at the bottom layer of physics. It's a form of emergence but it is different from the approximate forms that we can unpack.
I think symmetry emerges at that bottom level and is then spontaneously broken or partly hidden as space and time emerge higher up.