Advances in nanomechanics within recent years have demonstrated an always expanding range of devices, from top-down structures to appealing bottom-up MoS2 and graphene membranes, used for both sensing and component-oriented applications. One of the main concerns in all of these devices is frequency noise, which ultimately limits their applicability. This issue has attracted a lot of attention recently, and the origin of this noise remains elusive to date. In this article we present a very simple technique to measure frequency noise in nonlinear mechanical devices, based on the presence of bistability. It is illustrated on silicon-nitride high-stress doubly clamped beams, in a cryogenic environment. We report on the same T/f dependence of the frequency noise power spectra as reported in the literature. But we also find unexpected damping fluctuations, amplified in the vicinity of the bifurcation points; this effect is clearly distinct from already reported nonlinear dephasing and poses a fundamental limit on the measurement of bifurcation frequencies. The technique is further applied to the measurement of frequency noise as a function of mode number, within the same device. The relative frequency noise for the fundamental flexure δf/f0 lies in the range 0.5-0.01 ppm (consistent with the literature for cryogenic MHz devices) and decreases with mode number in the range studied. The technique can be applied to any type of nanomechanical structure, enabling progress toward the understanding of intrinsic sources of noise in these devices. © Copyright 2018 American Chemical Society.