Joe,
As children, we certainly only see the surface and many people don't get much beyond it, but some of us are curious about what goes on under the surface and what makes things tick. For instance, as a child, we might look out across the surface of the ocean and it doesn't occur to us of what might be beneath the surface, though we quickly get the sense not to go too far in.
As for seeing time in reverse, by looking only at the surface, a similar situation occurs when we look up at the sky and see the sun and the stars flowing across the sky, from east to west. Brilliant and predictively accurate geometric models of these actions, called epicycles, were constructed, based on these observations. When we tried to explain these models, by proposing crystalline spheres to carry the celestial bodies, we overlooked an important detail under our very feet. That it is this ground we are standing on, that is spinning the other way. West to east.
Spacetime is the modern version of those crystalline spheres. As a physical explanation for the very effective mathematical model of Special and General Relativity, it doesn't take into account that this narrative sequence, measured as duration, is effect, not cause. The present doesn't move, the events do.