Proceedings of the Annual Meeting of the Cognitive Science Society

Machine learning offers techniques for predicting behavior, but has limited capability to use behavior, context, and domain knowledge to infer intentions and predict behavior trajectory (i.e., sequence of directed actions towards reaching goal) of humans in real time to assist in strategic planning. Some existing models of human goal inference (Zhi-Xuan, Mann, Silver, Tenenbaum, & Manshinghka, 2020) possess some of these capabilities, however, these models are typically evaluated in toy worlds, which limits our understanding of how these approaches would generalize to real-world domains. Here, we introduce a novel integration of Bayesian goal inference and a deep learning model used for joint estimation (OpenPose) to address this generalizability problem and evaluate how this integrated approach can infer the goals of humans in real-world environments. This exploration provides an opportunity to evaluate capabilities required for real-time goal inference in real world environments and can highlight benefits and limitations informing future research.