Computing elastic properties is essential for designing materials with desirable mechanical properties. Ab initio methods, such as density functional theory (DFT), are commonly used to compute elastic properties, guiding experimental alloy design. Universal machine learning interatomic potentials (MLIPs) such as Universal Models for Atoms (UMA) have emerged as efficient and generalizable surrogates for DFT to model molecules, materials and catalysts. MLIPs are typically evaluated based on the accuracy of predicting energies and forces in atomistic simulations, but these metrics do not always translate to more accurate physical property prediction. Instead, we evaluate the accuracy of UMA in predicting relevant elastic properties, such as shear and bulk moduli. In this work, we assess the accuracy using a dataset from the Materials Project, comprising 12,000 structures, including unary, binary, ternary, and higher-order alloys. Our analysis shows that the accuracy in predicting the elastic properties is strongly influenced by the smoothness of the potential energy surface (PES) constructed by the MLIP. We find that the eSEN model architecture of UMA, which was designed to have a smoothly-varying PES, has the lowest errors with 5.6 and 7.6 GPa on the bulk and shear moduli, respectively. This work introduces a new experimentally relevant metric for evaluating the accuracy of MLIPs and highlights their potential for elastic property calculation, advancing machine learning in computational materials science.