Proceedings of the Annual Meeting of the Cognitive Science Society

It remains unclear whether multimedia facilitates or impairsknowledge acquisition. Here we examined whether subtitlesand video content facilitate comprehension of documentariesconsisting of statements of facts and whether thecomprehension depends on participants’ cognitive abilitiesand eye movement strategies during video watching. Wefound that subtitles facilitated comprehension regardless ofparticipants’ cognitive abilities or eye movement strategiesfor video watching. In contrast, with video content but notsubtitles, comprehension depended on participants’ auditoryworking memory, task switching ability, and eye movementstrategy. Through the Eye Movement analysis with HiddenMarkov Models (EMHMM) method, we found that acentralized (looking mainly at the screen center) eyemovement strategy predicted better comprehension asopposed to a distributed strategy (with distributed regions ofinterest) after contributions from cognitive abilities werecontrolled. Thus, whether video content facilitatescomprehension of documentaries depends on the viewers’ eyemovement strategy in addition to cognitive abilities.