Tensioned graphene membranes are of interest both for fundamental physics and for applications ranging from water filtration to nanomechanical resonators. It is generally assumed that these membranes have a stretching modulus of about 340 N/m and a negative, temperature-independent thermal expansion coefficient due to transverse phonon modes. In this paper, we study the two-dimensional Young's modulus and thermal expansion of graphene as functions of temperature by using laser interferometry to detect the static displacement of the membrane in a cryostat. Surprisingly, we find that the modulus decreases strongly with increasing temperature, which leads to a positive temperature-dependent thermal expansion coefficient. We show that the thermally rippled membrane theory is not consistent with our data, while the effects of surface contaminants typically present on these membranes may explain the observed behavior. Our experiments undermine long-standing assumptions about tensioned two-dimensional membranes, but are consistent with puzzling behavior observed in previous experiments on graphene resonators. © 2018 American Physical Society.