UC Santa Cruz

Open AI's Universe, intended to foster the development of general AI agents that could use a computer like humans do, fell short in a number of ways. As a result, attention shifted to smaller sets of environments such as Gym Retro or the Arcade Learning Environment. My thesis shows a way to overcome several of the limitations in the design of Universe by abstracting the execution of an interactive application in a full-system virtual machine. By using virtual machines, we regain the broad scope of the original Universe project, as well as the ability to pause, save, and restore at arbitrary moments of interaction, allowing the usage of several sophisticated exploration algorithms. I describe an API that implements the virtual machine strategy using Oracle's Virtual Box and ways to extend it with a framework for sensors and input actions that allow a broad range of manipulations of the virtual machines. I then walk through experiments designed to characterize the interaction latency and jitter associated with my implementation, as well as analyze the space costs associated with these large VMs and their associated save points. Finally, I demonstrate the utility of this implementation by showing automated play of two games.