We use information geometry in which the local distance between models measures their distinguishability from data to quantify the flow of information under the renormalization group. We show that information about relevant parameters is preserved with distances along relevant directions maintained under flow. By contrast, irrelevant parameters become less distinguishable under the flow with distances along irrelevant directions contracting according to renormalization group exponents. We develop a covariant formalism to understand the contraction of the model manifold. We then apply our tools to understand the emergence of the diffusion equation and more general statistical systems described by a free energy. Our results give an information-theoretic justification of universality in terms of the flow of the model manifold under coarse graining. © 2018 American Physical Society.