We study mechanisms of vortex nucleation in Nb3Sn superconducting RF (SRF) cavities using a combination of experimental, theoretical, and computational methods. Scanning transmission electron microscopy imaging and energy dispersive spectroscopy of some Nb3Sn cavities show Sn segregation at grain boundaries in Nb3Sn with Sn concentration as high as ∼35 at. % and widths ∼3 nm in chemical composition. Using ab initio calculations, we estimate the effect excess tin has on the local superconducting properties of the material. We model Sn segregation as a lowering of the local critical temperature. We then use time-dependent Ginzburg-Landau theory to understand the role of segregation on magnetic vortex nucleation. Our simulations indicate that the grain boundaries act as both nucleation sites for vortex penetration and pinning sites for vortices after nucleation. Depending on the magnitude of the applied field, vortices may remain pinned in the grain boundary or penetrate the grain itself. We estimate the superconducting losses due to vortices filling grain boundaries and compare with observed performance degradation with higher magnetic fields. We estimate that the quality factor may decrease by an order of magnitude (1010 to 109) at typical operating fields if 0.03% of the grain boundaries actively nucleate vortices. We additionally estimate the volume that would need to be filled with vortices to match experimental observations of cavity heating. © 2021 American Physical Society.