Excitons are prevalent in semiconductors and insulators, and their binding energies are critical for optoelectronic applications. The state-of-the-art method for first-principles calculations of excitons in extended systems is the ab initio GW-Bethe-Salpeter equation (BSE) approach, which can require a fine sampling of reciprocal space to accurately resolve solid-state exciton properties. Here we show, for a range of semiconductors and insulators, that the commonly employed approach of uniformly sampling the Brillouin zone can lead to underconverged exciton binding energies, as impractical grid sizes are required to achieve adequate convergence. We further show that nonuniform sampling of the Brillouin zone, focused on the region of reciprocal space where the exciton wave function resides, enables efficient rapid numerical convergence of exciton binding energies at a given level of theory. We propose a well-defined convergence procedure, which can be carried out at relatively low computational cost and which in some cases leads to a correction of previous best theoretical estimates by almost a factor of 2, qualitatively changing the predicted exciton physics. These results call for the adoption of nonuniform sampling methods for ab initio GW-BSE calculations and for revisiting previously computed values for exciton binding energies of many systems.