Thank you John for your comments.
Your criticisms are certainly not new, and I think they all are misguided. Everything in reality is connected to everything else. Finite and Infinite are interrelated.
Here are some evidences:
1. Higher axioms of infinity are not only relevant but also NECESSARY for finite mathematics.
See for example:
(i) Harvey Friedman "Necessary Uses of Abstract Set Theory in Finite Mathematics " Advances in Mathematics, Volume 60, issue 1, 1986.
(ii) Harvey Friedman, "Finite functions and the necessary use of large cardinals", Annals of Mathematics, 148; pg. 803-893, 1998.
(iii) Richard Laver, "On the algebra of elementary embeddings of a rank into itself", Advances in Mathematics, Volume 110, issue 2, pg. 334-346, 1995
2. There is the robust hierarchy of higher axioms of infinity.
See for example:
(i) John Steel, "Godel's Program", Interpreting Godel, pg. 153-179 (2014), Cambridge University Press.
(ii) Hugh W. Woodin, "The realm of the infinite" Infinity, new research frontiers, pg. 89-118 (2011) Cambridge University Press.
3. There is growing literature that shows the connection between set theory, higher axioms of infinity and physics.
See for example:
(i) Stanislaw Ulam" Combinatorial Analysis in Infinite Sets and Some Physical Theories", SIAM Review, Vol. 6, No. 4 (1964), pp. 343-355.
(ii) Marian Boykan Pour-El and Ian Richards: "Noncomputability in Analysis and Physics: A Complete Determination of the Class of Noncomputable Linear Operators", Advances in Mathematics 48, 44-74 (1983).
(iii) Robert Van Wesep, "Hidden variable in quantum mechanics: Generic models, set theoretic forcing, and emergence of probability", Annals of Physics 321 (2006) 2453-2490.
(iv) Paweł Klimasara, Jerzy Król, "Remarks on mathematical foundations of quantum mechanics" Acta Physica Polonica B, Volume 46. 2015.
(v) Toby Cubit, David P. Garcia, Michael M, Wolf, "Undecidability of the spectral gap", Nature 528, 2015.
(vi) Paul Corazza, "The Axiom of Infinity, Quantum Field Theory, Large Cardinals", The Review of Symbolic Logic, 2018.
4. There is the possibility of overcoming the Turing barrier (i.e. computing non-computable functions) via Hypercomputation.
See for example:
(i) Istvan Nemeti and Hajnal Andreka, "New Physics and Hypercomputation", SOFSEM 2006: Theory and Practice of Computer Science, Springer-Verlag.
(ii) Istvan Nemeti and David Gyula : "Relativistic Computers and the Turing Barrier" Journal of Applied Mathematics and Computation, Volume 178, issue 1, 2006.
5. There is the possibility that our physical universe is infinite. If the theory of cosmic inflation is correct, a multiverse is inevitable. In a multiverse, one cannot avoid infinity.
Everything in reality is connected to everything else. Finite and Infinite are interrelated.