We give a bird's-eye view of the plastic deformation of crystals aimed at the statistical physics community, as well as a broad introduction to the statistical theories of forced rigid systems aimed at the plasticity community. Memory effects in magnets, spin glasses, charge density waves, and dilute colloidal suspensions are discussed in relation to the onset of plastic yielding in crystals. Dislocation avalanches and complex dislocation tangles are discussed via a brief introduction to the renormalization group and scaling. Analogies to emergent scale invariance in fracture, jamming, coarsening, and a variety of depinning transitions are explored. Dislocation dynamics in crystals challenge nonequilibrium statistical physics. Statistical physics provides both cautionary tales of subtle memory effects in nonequilibrium systems and systematic tools designed to address complex scale-invariant behavior on multiple length scales and timescales. © 2017 by Annual Reviews. All rights reserved.