Proceedings of the Annual Meeting of the Cognitive Science Society

We introduce a novel experimental paradigm for studying multi-modal integration in causal in-ference. Our experiments feature a physically realistic Plinko machine in which a ball is droppedthrough one of three holes and comes to rest at the bottom after colliding with a number of ob-stacles. We develop a hypothetical simulation model which postulates that people figure out whathappened by integrating visual and auditory evidence through mental simulation. We test themodel in a series of three experiments. In Experiment 1, participants only receive visual infor-mation and either predict where the ball will land, or infer in what hole it was dropped based onwhere it landed. In Experiment 2, participants receive both visual and auditory information – theyhear what sounds the dropped ball makes. We find that participants are capable of integratingboth sources of information, and that the sounds help them figure out what happened. In Exper-iment 3, we show strong cue integration: even when vision and sound are individually completelynon-diagnostic, participants succeed by combining both sources of evidence.