Animals can leverage prior experience to guide adaptive decision making. To decide where to forage, for instance, an animal may recall previous locations and internally simulate paths to take next. These functions are thought to rely on the hippocampus, a brain structure long implicated in learning, memory, and navigation. Accordingly, hippocampal neural activity can represent an animal’s current position, as well as generate representations of alternative possible locations. These representations of alternative “non-local” possibilities are hypothesized to enable internal simulation of previous experiences, alternative pasts, and potential futures to support cognition and, in turn, experience-guided decision making. However, it remains unclear whether or how internally generated hippocampal non-local representations are regulated during active behavior depending on changing cognitive needs for learning about and deciding among alternatives. In this work, I first synthesize evidence describing hippocampal non-local representations that suggests that they are well-suited to serve a wider range of cognitive abilities than previously thought. This work advances the idea that hippocampal function is well characterized not only by its representation of actual experience, but also by its regular representation of alternatives to actual experience. I then present experimental findings that show that the hippocampus generates representations of a wide range of spatial possibilities during active navigation, and that representations of these distinct possibilities are distinctly modulated with learning and decision making in a complex and dynamic foraging environment. These findings indicate that the brain regulates the generation of alternatives in the hippocampus to meet momentary cognitive demands for adaptive behavior.