The discovery of drug-like molecules that bind pockets in proteins that are not present in crystallographic structures yet exert allosteric control over activity has generated great interest in designing pharmaceuticals that exploit allosteric effects. However, there have only been a small number of successes, so the therapeutic potential of these pockets--called hidden allosteric sites--remains unclear. One challenge for assessing their utility is that rational drug design approaches require foreknowledge of the target site, but most hidden allosteric sites are only discovered when a small molecule is found to stabilize them. We present a means of decoupling the identification of hidden allosteric sites from the discovery of drugs that bind them by drawing on new developments in Markov state modeling that provide unprecedented access to microsecond- to millisecond-timescale fluctuations of a protein's structure. Visualizing these fluctuations allows us to identify potential hidden allosteric sites, which we then test via thiol labeling experiments. Application of these methods reveals multiple hidden allosteric sites in an important antibiotic target--TEM-1 β-lactamase. This result supports the hypothesis that there are many as yet undiscovered hidden allosteric sites and suggests our methodology can identify such sites, providing a starting point for future drug design efforts. More generally, our results demonstrate the power of using Markov state models to guide experiments.