Proceedings of the Annual Meeting of the Cognitive Science Society

Gaze interaction behavior recognition (GIBR) is essential for understanding social cognition, yet existing multimodal methods often rely on implicit affect inference and are limited to short-term temporal modeling. We propose Mamba-GazeNet, a multimodal spatiotemporal framework that integrates explicit emotion-calibrated gaze graph construction with efficient long-sequence modeling via Mamba. Facial affect vectors extracted from a pre-trained emotion recognizer are combined with interpersonal geometry to build emotion-aware gaze graphs, while a gated fusion strategy suppresses unreliable emotional cues to ensure structural robustness. A graph embedding layer aggregates intra-frame spatial relations, and a Mamba-based temporal module models long-range interaction dynamics with linear complexity. This design jointly captures affective cues, dynamic gaze patterns, and evolving social relations. Experiments on VACATION and the GIBR benchmark suite demonstrate that Mamba-GazeNet achieves state-of-the-art performance in both full-category recognition and single-category generalization, highlighting the importance of explicit emotion supervision and scalable temporal reasoning for reliable social behavior understanding.