Skip to main content
eScholarship
Open Access Publications from the University of California

MoEEG: A Sparse Mixture-of-Experts Transformer for Universal EEG Representation Learning

Creative Commons 'BY' version 4.0 license
Abstract

Electroencephalography (EEG) provides a non-invasive and temporally precise window into neural dynamics, yet learning transferable representations across subjects, paradigms, and electrode layouts remains challenging. Existing EEG foundation models typically rely on patch-wise tokenization and dense Transformer blocks, which may weaken fine-grained temporal continuity and provide limited adaptability to heterogeneous neural patterns. We propose MoEEG, a sparse Mixture-of-Experts (MoE) Transformer for generalizable EEG representation learning. MoEEG combines global-local temporal embedding, electrode-aware channel embedding, a dual-path EEGAttention module for temporal and spatial dependency modeling, and sparse expert routing for adaptive representation learning. Linear-probing evaluations on P300, ERN, and motor imagery benchmarks show that MoEEG consistently outperforms BIOT, LaBraM, and EEGPT, with balanced-accuracy improvements over EEGPT of 2.94, 1.69, 0.72, and 1.43 percentage points on PhysioP300, KaggleERN, BCIC-2A, and BCIC-2B, respectively. These results suggest that sparse expert routing and EEG-specific attention can improve cross-task EEG representation learning. Our open source code is available at https://github.com/YuuSenW/MoEEG.git.