Neural correlates of affective context in facial expression analysis: A simultaneous EEG-fNIRS study
2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp 820-824
Dec 2015
We present a framework to measure the correlation between spontaneous human facial affective expressions and relevant brain activity. The affective states were registered from the video capture of facial expression and related neural activity was measured using wearable and portable neuroimaging systems: functional near infrared spectroscopy (fNIRS), electroencephalography (EEG) to asses both hemodynamic and electrophysiological responses. The methodology involves the simultaneous detection and comparison of various affective expressions by multi-modalities and classification of spatio-temporal data with neural signature traits. The experimental results show strong correlation between the spontaneous facial affective expressions and the affective states related brain activity. We propose a multimodal approach to jointly evaluate fNIRS signals and EEG signals for affective state detection. Results indicate that proposed method with fNIRS+EEG improves performance over fNIRS or EEG only approaches. These findings encourage further studies of the joint utilization of video and brain signals for face perception and brain-computer interface (BCI) applications.
2
16
- Neural correlates of affective context in facial expression analysis: A simultaneous EEG-fNIRS study
- Yanjia Sun - New Jersey Institute of TechnologyHasan Ayaz - Drexel University, School of Biomedical Engineering, Science, and Health SystemsAli N Akansu - New Jersey Institute of Technology
- 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp 820-824
- 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP) (Orlando, Florida, United States, 14 Dec 2015–16 Dec 2015)
- IEEE
- 5
- Conference proceeding
- English
- School of Biomedical Engineering, Science, and Health Systems
- 2-s2.0-84964720814
- 991014877685004721