An accurate estimation of driver vigilance is crucial for reducing fatigue-related incidents and traffic accidents. Despite advances in the field of fatigue detection, effective utilization of multimodal information remains a major challenge. Additionally, prevalent methodologies predominantly focus on local features, overlooking the importance of global features in this context. To solve the above problems, we propose the deep channel attention transformer (DCAT) model, which can effectively utilize multimodal information and extract local-global features for fatigue detection regression tasks. We first introduce a novel multimodal approach that integrates electroencephalography (EEG) and electrooculogram (EOG) data, capitalizing on their complementary strengths to enhance the understanding and assessment of fatigue states. Then, the DCAT model utilizes multimodal information by extracting local and global features using channel attention and transformer encoder modules, respectively. Our evaluation of the SEED-VIG and SADT public datasets showcases the model's superior performance compared to that of the state-of-the-art baselines.

Currently, emotion recognition based on electroencephalograms (EEGs) has a wide range of applications. Although many approaches have been proposed for automatic emotion recognition with favorable performance, there are still several challenges: (1) how to sufficiently model the long- and short-term temporal feature discrepancies and redundant spatial information of EEGs and (2) how to alleviate the negative impact of the ambiguity of emotion classes. To tackle these issues, we propose the CSET-CCA, a novel framework for EEG-based emotion recognition. The feature extractor of this model combines the 1D convolutional neural network (CNN), channel Squeeze-and-Excitation (SE) module and transformer. It can extract the temporal features of EEG signals from local and global perspectives and select the critical channels in emotion recognition. Moreover, to adaptively perceive the confusion degrees of classes and increase the model's attention on confusing emotion classes, we design class confusion-aware (CCA) attention. We evaluate the CSET-CCA with the SEED and SEED-V datasets. The experimental results show that the proposed approach outperforms state-of-the-art methods.

Prognostic assessment of patients with disorders of consciousness (DoC) remains one of the most challenging problems in contemporary medicine. The long treatment cycle and high costs of treatment are heavy burdens to our society. In this paper, we use deep network to investigate potential indicators of consciousness within brain signals of DoC patients. In the experiments, we study P300 and resting-state Electroencephalogram (rs-EEG) signals of 22 DoC patients to investigate neural correlation between brain signals and the improvement of consciousness. Synergistic integration of P300 and rs-EEG signals demonstrated superior predictive proficiency for cross-subject and cross-paradigm prognosis in DoC, achieving an accuracy of 81.1%. Our investigation is the first known to the literature to combine P300 and rs-EEG signals for analyzing DoC. This novel approach leverages advanced neural network models to elucidate the complex neural patterns associated with DoC, setting a precedent for future research in the field.

Electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS), both portable and non-invasive, enhance brain-computer interface (BCI) performance by integrating their spatial and temporal benefits when combined together. However, the fusion of these two signals still faces challenges. To fully unitize the complementarity of EEG and fNIRS for improved performance in EEG-fNIRS BCI, we propose an EEG-fNIRS fusion network based on end-to-end mutual information learning, named EFMLNet. In the model, EEG and fNIRS data are fed into their respective feature extractors for the extraction of temporal and spatial information. Furthermore, their complementary information is fused by two parallel mutual learning modules. We conducted classification experiments on a publicly available BCI dataset based on motor imagery (MI) task and achieved a cross-subject classification accuracy of 71.52%. This result surpasses the performance of most existing fusion methods and demonstrates the potential for real-time hybrid BCI systems.

Sleep staging holds significant importance in clinical medicine, aiding in the diagnosis of various disorders related to sleep and cognition. However, manually annotating a large amount of sleep data is time-consuming and labor-intensive, making it difficult to achieve. Efficiently utilizing these unannotated data poses a challenging task. We propose a novel self-supervised learning method with Temporal-split Contrastive and Electrode Autoencoder (TsC-EA) for sleep staging. We demonstrate that our method achieves state-of-the-art performance in self-supervised learning on SleepEDF and MASS-SS3. Moreover, experimental results indicate that our method can surpass the performance of supervised learning methods using only 10% of labeled data. Additionally, we explore the application of self-supervised learning in patients with disorder of consciousness. It can assist in diagnosing the severity of DoC through analysis of sleep staging. Staging the sleep patterns of patients with disorders of consciousness can help in diagnosing the severity of their condition.

Emotion recognition is crucial for enhancing human-computer interaction. Due to considerable individual differences in emotion manifestation, traditional models do not adapt well to new individuals. Moreover, existing algorithms typically focus on identifying a single emotion, overlooking intrinsic connections among multiple emotions. Therefore, we propose a multi-task adversarial domain adaption (MADA) model for EEG-based emotion recognition. First, domain matching is employed to identify the most similar individual from the dataset as the source domain, alleviating individual differences and reducing training time. Subsequently, multi-task learning is utilized to simultaneously classify multiple emotions, capturing their intrinsic connections. Finally, adversarial domain adaption is applied to learn the individual differences between the source and target domains. Cross-subject experiments on the DEAP dataset indicate that our model achieves accuracies of 78.08%, 68.36%, and 69.64% on the valence, arousal, and dominance, respectively, surpassing state-of-the-art methods. This indicates the effectiveness of our model in recognizing multi-dimensional emotions.