Journal article
Speech Recognition via fNIRS Based Brain Signals
Frontiers in neuroscience, v 12, pp 695-695
09 Oct 2018
PMID: 30356771
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
In this paper, we present the first evidence that perceived speech can be identified from the listeners' brain signals measured via functional-near infrared spectroscopy (fNIRS)—a non-invasive, portable, and wearable neuroimaging technique suitable for ecologically valid settings. In this study, participants listened audio clips containing English stories while prefrontal and parietal cortices were monitored with fNIRS. Machine learning was applied to train predictive models using fNIRS data from a subject pool to predict which part of a story was listened by a new subject not in the pool based on the brain's hemodynamic response as measured by fNIRS. fNIRS signals can vary considerably from subject to subject due to the different head size, head shape, and spatial locations of brain functional regions. To overcome this difficulty, a generalized canonical correlation analysis (GCCA) was adopted to extract latent variables that are shared among the listeners before applying principal component analysis (PCA) for dimension reduction and applying logistic regression for classification. A 74.7% average accuracy has been achieved for differentiating between two 50 s. long story segments and a 43.6% average accuracy has been achieved for differentiating four 25 s. long story segments. These results suggest the potential of an fNIRS based-approach for building a speech decoding brain-computer-interface for developing a new type of neural prosthetic system.
Metrics
Details
- Title
- Speech Recognition via fNIRS Based Brain Signals
- Creators
- Yichuan Liu - School of Biomedical Engineering, Drexel University, Science and Health SystemsHasan Ayaz - School of Biomedical Engineering, Drexel University, Science and Health Systems
- Publication Details
- Frontiers in neuroscience, v 12, pp 695-695
- Publisher
- Frontiers Media S.A
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- School of Biomedical Engineering, Science, and Health Systems
- Web of Science ID
- WOS:000446836400001
- Scopus ID
- 2-s2.0-85055273382
- Other Identifier
- 991014878479904721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Neurosciences