We analyze the out-of-time-order correlation functions of a solvable model of a large number N of M-component quantum rotors coupled by Gaussian-distributed random, infinite-range exchange interactions. We focus on the growth of commutators of operators at a temperature T above the zero temperature quantum critical point separating the spin-glass and paramagnetic phases. In the large N,M limit, the squared commutators of the rotor fields do not display any exponential growth of commutators, in spite of the absence of any sharp quasiparticlelike excitations in the disorder-averaged theory. We show that in this limit, the problem is integrable and points out interesting connections to random-matrix theory. At leading order in 1/M, there are no modifications to the critical behavior but an irrelevant term in the fixed-point action leads to a small exponential growth of the squared commutator. We also introduce and comment on a generalized model involving p-pair rotor interactions. © 2020 American Physical Society.