I agree that mathematics/information is physical, and want to note something I thought of while reading your essay (that I'm sure you already thought of long before me).
It's easy to argue endlessly about whether mathematics or mathematical objects exist in some Platonic realm. But our ideas about mathematical objects are most certainly physical, in the following sense. When I think of a perfect circle, the neurons in my brain fire a certain way: there is some physical configuration of my brain associated with thinking about a perfect circle, or any other mathematical object I'm familiar with. When other people think about perfect circles, their neurons might fire in a different way, so that the 'same' information is stored differently. Regardless of how each of us stores that information, we can all access and output it in different forms, like as lines on paper or algebraic equations on a blackboard. Someone else can learn about circles by studying these different representations, and store their own representation of one. The mathematical idea of a perfect circle physically exists as the collection of all instantiations of the associated information.
How much energy is spent every day on the storage, retrieval, and communication of some mathematical idea? I wonder.
It's a little mind-bendy to think about abstract ideas having causal power in a physical universe. But it makes more sense to me when I remember that information, mathematical or otherwise, must be stored in some physical configuration of matter. Only physical things can have causal power in a physical universe, I think.
Appreciate the beautiful, deliberate writing. The beginning felt like a novel, and I got 'hooked' for the rest of your essay.