Why quantum mechancs? We really do not know. As I have been studying this it appears that quantum mechanics is really a logical system of gates where there are certain topological properties to the lattice operations that deviate from Boolean logic. The two slit experiment is a sort of topological problem with loops that are not contractible to a point. There is homotopy associated with this. Well then why this? Why could not the foundations be and L^4 measure space instead of L^2, or why could it not have been ... , instead of this? The questions have no conceivable answer.
Quantum mechanics is completely deterministic as it is. Quantum mechanics predicts the evolution of probability amplitudes that sum into a wave function or state vector with complete determinism. That quantum mechanics is unitary is equivalent to saying it is deterministic. The probabilities emerge when one makes a measurement, for the modulus square of those amplitudes give the probability of a certain outcome. This is the bit that is not deterministic --- the measurement process. This forms the basis of the so called measurement problem.
I maintain that a quantum outcome is not objective, but is rather subjective. From the perspective of the Everett Many Worlds Interpretation (MWI) a measurement outcome is a splitting of the world according to the amplitudes of the system. The reservoir of states in the measurement process form an eigenbasis corresponding to the entanglement with the system measured. This in a nutshell is the so called einselection mechanism, which is just a massive entanglement process. We can then tell how the outcomes emerge as classical-like probabilities. We can't however determine which outcome actually obtains. In a subjective perspective the observer is carried off into the various split worlds, and our conscious world line simply takes along one outcome. Consciousness is an epiphenomenon that generates this illusion. The illusion is carried off along the other eigenbranches of the world, but our particular conscious narrative does not include them. These are included on other conscious narratives or world lines.
Why quantum mechanics exists is somehow tied into the question of what is the relationship between mathematics and physics. Tegmark has a very speculative conjecture about this, but I am rather skeptical. I am not sure how one can determine if mathematics is an ontological aspect of physics. There is no mathematical proof of it, nor is there a prospect for some empirical verification of it. The idea is permanently a metaphysical conjecture that is hopelessly outside of both science and mathematics. The same of course holds for the converse of this which is for a Platonic reality or mathematics as an objective system outside of physical reality. This matter is extended further if consciousness is included as something fundamental. No matter which take you have on this the question or proposed ideas are metaphysical. So far there does not exist a decision procedure system for metaphysics.
In the end we are left with Garrison Keillor's take on this as "life's persistent questions." --- " In a city that knows how to keep its secrets, at the 12th floor of the Atlas building one man searches for answers to life's persistent questions; Guy Noir, private eye."
LC