Human Casual Transfer: Challenges for Deep Reinforcement Learning
Skip to main content
eScholarship
Open Access Publications from the University of California

Human Casual Transfer: Challenges for Deep Reinforcement Learning

Abstract

Discovery and application of causal knowledge in novel problem contexts is a prime example of human intelligence. As new in- formation is obtained from the environment during interactions, people develop and refine causal schemas to establish a parsimo- nious explanation of underlying problem constraints. The aim of the current study is to systematically examine human abil- ity to discover causal schemas by exploring the environment and transferring knowledge to new situations with greater or differ- ent structural complexity. We developed a novel OpenLock task, in which participants explored a virtual “escape room” environ- ment by moving levers that served as “locks” to open a door. In each situation, the sequential movements of the levers that opened the door formed a branching causal sequence that began with either a common-cause (CC) or a common-effect (CE) struc- ture. Participants in a baseline condition completed five trials with high structural complexity (i.e., four active levers). Those in the transfer conditions completed six training trials with low structural complexity (i.e., three active levers) before completing a high-complexity transfer trial. The causal schema acquired in the transfer condition was either congruent or incongruent with that in the transfer condition. Baseline performance under the CC schema was superior to performance under the CE schema, and schema congruency facilitated transfer performance when the congruent schema was the less difficult CC schema. We com- pared between-subjects human performance to a deep reinforce- ment learning model and found that a standard deep reinforce- ment learning model (DDQN) is unable to capture the causal ab- straction presented between trials with the same causal schema and trials with a transfer of causal schema.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View