We explore matrix product state approximations to wave functions which have spontaneously broken symmetries or are critical. We are motivated by the fact that symmetries, and their associated conservation laws, lead to block-sparse matrix product states. Numerical calculations which take advantage of these symmetries run faster and require less memory. However, in symmetry-broken and critical phases the block-sparse ansatz yields less accurate energies. We characterize the role of conservation laws in matrix product states and determine when it is beneficial to make use of them. © 2022 American Physical Society.