Great essay.
I think math is discovered. I think you present a very good analysis of the comparison of discovered vs. invented. I'll spend more time thinking about your `invented' arguments.
The following addresses your objections to `discovered' (``Math is a part of nature...".)
A There is a need to discover the truth of how nature works to advance our survival. There is much more to learn about the universe, math and physics. We don't have different math formulas for the same physics phenomena. We have different assumptions for the same phenomena. OR, the formulas produce the same predictions with an easier formulation. Liebniz postulated if the different formulas produce the same results, they are the same.
B Even math starts with axioms. So, derivation of principles is not done in physics or math.
C We have many mysteries. Saying they have no solution is inaccurate and misleading. More accurate is we haven't discovered (or invented) the math yet. For example, Fourier analysis was unknown but the Greeks had `harmony of the spheres' and circles within circles (Ptolomey).
D Your 4th objection is really the crux of the whole issue. Infinity is not a number. It means unbounded or the increase without limit. Whether the universe is unbounded or not is currently a metaphysical issue. I consider it bounded and flat (see my essay on how). Thanks to the essay by Ojo and conversation with him, I have come to think that division is an unnatural operation. I think irrational numbers are invented and therefore are not natural and not valid math. Its purpose is to be an inverse of multiplication, but multiplication is repetitive addition. Therefore, the inverse of multiplication is repetitive subtraction. It solves Zeno's paradox. That is, the invented parts of math are invalid and not natural.
Your section on Patterns and Regularities argues for math being a part of nature. Thanks also for the recognition of the fractal nature of the universe.
Statistics may be an invention. I take the nature of physics to be cause and effect. Statistics addresses a situation where a pattern in the data has been noted and no cause-effect model has yet been developed. For example, QM may indicate the type of cause-effect model such as the Bohm Interpretation (I've researched this wit a model of photon interference). I suggest Group models are like the periodic table in the 19th century. A pattern was recognized; the table was constructed based on properties, holes in the table predicted new elements and their properties. Further, the early 20th century saw the development of a cause-effect (structure) model of why the periodic table worked. The table itself was invented, but the cause-effect was natural. The same is true of the group model where the holes predicted the properties of undiscovered particles.
I particularily like the `Beautiful but wrong' section. I suggest all our current models are `wrong' in the sense of limited.
Your essay is one of the very few that addresses the topic. Well, done!