UCLA

Probabilistic modeling and reasoning are central tasks in artificial intelligence and machine learning. A probabilistic model is a rough description of the world: the model-builder attempts to capture as much detail about the world’s complexities as she can, and when no more detail can be given the rest is left as probabilistic uncertainty. Once constructed, the goal of a model is to perform automated inference: compute the probability that some particular fact is true about the world. It is natural for the model-builder to want a flexible expressive language – the world is a complex thing to describe – and over time this has led to a trend of increasingly powerful modeling languages. This trend is taken to its apex by probabilistic programming languages (PPLs), which enable modelers to specify probabilistic models using the facilities of a full programming language. However, this expressivity comes at a cost: the computational cost of inference is in direct tension with the flexibility of the modeling language, and so it becomes increasingly difficult to design automated inference algorithms that scale to the kinds of systems that model builders want to create.

This thesis focuses on the central question: how can we design effective probabilistic programming languages that profitably trade-off expressivity and tractability for inference? The approach taken here is first to identify and exploit important structure that a probabilistic program may possess. The kinds of structure considered here are discrete program structure and symmetry. Programs are heterogeneous objects, so different parts of programs may exhibit different kinds of structure; in the second part of the thesis I show how to decompose heterogeneous probabilistic program inference using a notion of program abstraction. These contributions enable new applications of probabilistic programs in domains such as text analysis, verification of probabilistic systems, and classical simulation of quantum algorithms.