Proceedings of the Annual Meeting of the Cognitive Science Society

Empathy in human communication relies on a dual process: affective empathy and cognitive empathy. While existing multimodal dialogue systems excel at detecting emotions from audio-visual cues, they often struggle to reason about the underlying context. To bridge this gap, we propose an approach that integrates a large vision-language model to generate dialogue-oriented situation appraisals to simulate cognitive reasoning, serving as an explicit source of cognitive empathy. We introduce a multi-source attention mechanism with a learned gate to fuse this high-level situational understanding with low-level affective cues without compromising generation stability. Experiments on the MELD and MEDIC datasets demonstrate that disentangling and then integrating these two empathetic pathways significantly enhances response appropriateness, aligning better with human empathetic processes.