The development of a mental workload assessment system using psychophysiological signals

Yichuan Liu

doi:10.17918/D82M20

An accurate measure of mental workload level from psychophysiological signals has diverse neuroergonomic applications ranging from brain computer interfacing to improving the efficiency of human operators. The techniques currently used to decode mental workload from psychophysiological signals are not yet field deployable due to a number of challenges, including the long calibration time required before the mental workload decoder can be used. Current available techniques require the recording of lengthy signals to calibrate a workload-decoding model with machine learning before each use. This is mainly due to the challenge that psychophysiological signals vary considerably between different people and over time. The development of a "calibration-free" workload assessment system that functions without recording lengthy calibration data is needed. Improving the decoding accuracy presents a second challenge. Techniques have been developed to decode workload level mainly using the brain activity measured by an electroencephalogram (EEG). Although promising results have been reported, the decoding accuracy of these techniques needs to be improved. This thesis advances the field along two complementary paths by using a set of EEG, functional near-infrared spectroscopy (fNIRS), and physiological data, simultaneously recorded from 21 subjects during a working memory (n-back) task. First, it develops an alternative calibration approach to derive workload-decoding models as a step toward "calibration-free" workload assessment. Existing techniques calibrate workload-decoding models only using the data recorded from the target subject (the user). This thesis shows that workload-related patterns that are sufficiently common across different subjects can be found such that deriving workload-decoding models using the additional data recorded from other subjects improves the decoding accuracy. This is especially true when the amount of calibration data from the target subject is small. The results in this thesis provide the first evidence that machine learning from the data of multiple subjects outperforms learning from a single subject in terms of mental workload decoding accuracy. Second, this thesis acquires workload-relevant information from fNIRS and physiological measures such as heart rate variability and breath rate in addition to the most commonly adopted EEG for improving the accuracy of workload assessment. The results show that there is workload-related information that is complementary in EEG and fNIRS such that integrating the two outperforms using either alone in terms of workload decoding accuracy. The techniques developed in this thesis may have applications in neuroergonomics research and applications such as adaptive aiding systems that are designed to improve the efficiency and safety of human-machine systems during critical tasks.

The development of a mental workload assessment system using psychophysiological signals

Files and links (1)

Abstract

Metrics

Details

The development of a mental workload assessment system using psychophysiological signals

Files and links (1)

Abstract

Metrics

Details

Drexel University Social media