That doesn’t follow. Maybe the frequency of light is slowing down. Or maybe our clocks are speeding up.
We have no cosmological standards for time. We have no cosmological standards for distance. All of our standard are local standards. We base them on local phenomena, and then attempt to extrapolate them to the cosmos as a whole. But maybe there isn’t a possible time metric that can be extended throughout the entire cosmos. Maybe there isn’t a possible distance metric that can be extended thoughout the entire cosmos. Maybe the cosmos is a far stranger place than we take it to be.