The mechanical, structural and functional properties of crystals are determined by their defects, and the distribution of stresses surrounding these defects has broad implications for the understanding of transport phenomena. When the defect density rises to levels routinely found in real-world materials, transport is governed by local stresses that are predominantly nonlinear. Such stress fields however, cannot be measured using conventional bulk and local measurement techniques. Here, we report direct and spatially resolved experimental measurements of the nonlinear stresses surrounding colloidal crystalline defect cores, and show that the stresses at vacancy cores generate attractive interactions between them. We also directly visualize the softening of crystalline regions surrounding dislocation cores, and find that stress fluctuations in quiescent polycrystals are uniformly distributed rather than localized at grain boundaries, as is the case in strained atomic polycrystals. Nonlinear stress measurements have important implications for strain hardening, yield and fatigue. © 2016 Macmillan Publishers Limited, part of Springer Nature. All rights reserved.