Journal article
fNIRS-based classification of mind-wandering with personalized window selection for multimodal learning interfaces
JOURNAL ON MULTIMODAL USER INTERFACES, v 15(3), pp 257-272
Sep 2021
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
Automatic detection of an individual's mind-wandering state has implications for designing and evaluating engaging and effective learning interfaces. While it is difficult to differentiate whether an individual is mind-wandering or focusing on the task only based on externally observable behavior, brain-based sensing offers unique insights to internal states. To explore the feasibility, we conducted a study using functional near-infrared spectroscopy (fNIRS) and investigated machine learning classifiers to detect mind-wandering episodes based on fNIRS data, both on an individual level and a group level, specifically focusing on automated window selection to improve classification results. For individual-level classification, by using a moving window method combined with a linear discriminant classifier, we found the best windows for classification and achieved a mean F1-score of 74.8%. For group-level classification, we proposed an individual-based time window selection (ITWS) algorithm to incorporate individual differences in window selection. The algorithm first finds the best window for each individual by using embedded individual-level classifiers and then uses these windows from all participants to build the final classifier. The performance of the ITWS algorithm is evaluated when used with eXtreme gradient boosting, convolutional neural networks, and deep neural networks. Our results show that the proposed algorithm achieved significant improvement compared to the previous state of the art in terms of brain-based classification of mind-wandering, with an average F1-score of 73.2%. This builds a foundation for mind-wandering detection for both the evaluation of multimodal learning interfaces and for future attention-aware systems.
Metrics
Details
- Title
- fNIRS-based classification of mind-wandering with personalized window selection for multimodal learning interfaces
- Publication Details
- JOURNAL ON MULTIMODAL USER INTERFACES, v 15(3), pp 257-272
- Publisher
- SPRINGER; NEW YORK
- Number of pages
- 15
- Grant note
- This work was supported in part by the U.S. National Science Foundation under Grants NCS-1835307, CNS-1711773 and NCS-1835251.
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Drexel University
- Web of Science ID
- WOS:000537349200001
- Scopus ID
- 2-s2.0-85114310351
- Other Identifier
- 991021860660604721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Computer Science, Artificial Intelligence
- Computer Science, Cybernetics