Proceedings of the Annual Meeting of the Cognitive Science Society

Multi-modal Named Entity Recognition (MNER) and Multi-modal Relation Extraction (MRE) are critical for knowledge extraction. Existing methods often suffer from cross-modal confusion due to semantic misalignment and data imbalance. Inspired by predictive processing theory, i.e., the brain continuously refines cognition through active prediction and error minimization, we propose a Cognitive-inspired Meta-learning Adaptive model for Multi-modal Entity-relation extraction named CogMAME. It consists of five modules: (i) multi-grained representation learning, establishing a unified semantic space; (ii) class-adaptive feature augmentation, reinforcing signals for rare categories; (iii) meta weight hypernet, dynamically calibrating modality contributions; (iv) self-adversarial perturbation, improving stability via controlled noise; (v) multi-modal information extraction, refining outputs in a joint feature space. Extensive experimental results show that our model can achieve human-like extraction capabilities and obtain state-of-the-art performance on both MNER and MRE tasks, verifying its superiority and effectiveness.