Dear William T. Parsons,
I agree with your view on infinity. By adding finite numbers we can never reach infinity. We can introduce an infinite loop: go on adding infinitely, and it becomes a never ending process. Regarding the relation between physics and mathematics, I invite your attention to my essay: A physicalist interpretation of the relation between Physics and Mathematics
Hilbert's Hotel is based on wrong assumptions or axioms. (i). "Hilbert's Hotel consists of an infinite set of rooms". Never. Hilbert can start from zero, constructing rooms one by one but can never complete infinite rooms; here, construction is a never ending process. (ii). "an infinite set of buses arrives". Never. Buses keep coming; it is a never ending process. Hilbert's Hotel is actually an infinite loop (of finite processes). Thus in fact, there is no paradox.
I agree with you that infinity has no role in physics. However, I think the best method to avoid infinity is to 'finitise' one by one all the concepts and equations by suitably modifying the existing ones. Newton's straight-line motion, force- acceleration relation and equation for gravity, all lead us to infinity. Thus infinity crept into physics from the time of Newton (his third law is an exception). I have brought out a model devoid of infinity (refer: finitenesstheory.com).