Logo image
Back
Evaluation of interpretability for deep learning algorithms in EEG emotion recognition: A case study in autism
Journal article   Open access   Peer reviewed

Evaluation of interpretability for deep learning algorithms in EEG emotion recognition: A case study in autism

Juan Manuel Mayor Torres, Sara Medina-DeVilliers, Tessa Clarkson, Matthew D. Lerner and Giuseppe Riccardi
Artificial intelligence in medicine, v 143, 102545
Sep 2023
DOI: https://doi.org/10.1016/j.artmed.2023.102545
Featured in Collection :   UN Sustainable Development Goals @ Drexel
url
https://arxiv.org/pdf/2111.13208View
SubmittedarXiv.org - Non-exclusive license to distribute,  Open

Abstract

Autism Autism spectrum disorder Convolutional neural networks (CNN) Electroencephalography (EEG) Emotion recognition Explainable AI (XAI) Re-training RemOve-And-Retrain (ROAR) XAI methods
Current models on Explainable Artificial Intelligence (XAI) have shown a lack of reliability when evaluating feature-relevance for deep neural biomarker classifiers. The inclusion of reliable saliency-maps for obtaining trustworthy and interpretable neural activity is still insufficiently mature for practical applications. These limitations impede the development of clinical applications of Deep Learning. To address, these limitations we propose the RemOve-And-Retrain (ROAR) algorithm which supports the recovery of highly relevant features from any pre-trained deep neural network. In this study we evaluated the ROAR methodology and algorithm for the Face Emotion Recognition (FER) task, which is clinically applicable in the study of Autism Spectrum Disorder (ASD). We trained a Convolutional Neural Network (CNN) from electroencephalography (EEG) signals and assessed the relevance of FER-elicited EEG features from individuals diagnosed with and without ASD. Specifically, we compared the ROAR reliability from well-known relevance maps such as Layer-Wise Relevance Propagation, PatternNet, Pattern-Attribution, and Smooth-Grad Squared. This study is the first to bridge previous neuroscience and ASD research findings to feature-relevance calculation for EEG-based emotion recognition with CNN in typically-development (TD) and in ASD individuals. •The RemOve-And-Retrain (ROAR) methodology investigates and re-validates the XAI methods’ quantitative trustworthiness and certainty based on feature removal and a subsequent accuracy detriment. In this study, ROAR evidences clinically-consistent relevance-patterns in the more certain XAI methods during an EEG-based Face Emotion Recognition (FER) task including individuals diagnosed with and without Autism Spectrum Disorder (ASD).•ROAR is applied to a broad variety of well-known XAI methods, such as, PatternNet, Pattern-Attribution, Smooth-Grad Squared, and Layer-Wise Relevance Propagation (LRP). LRP shows significantly higher relevances in the late time components (e.g, early and middle Late Positive Potential) for ASDs eliciting negative emotions in comparison with Typically Developed (TD) individuals.•The more certain XAI methods found using ROAR are PatternNet, Pattern-Attribution, and LRP. LRP shows significantly different results between TD and ASD associated average relevance-maps, thus suggesting a different emotion information encoding between TD and ASD individuals.•During ROAR evaluation, binary masks patterns are defined in order to remove the potential-relevant features and re-validate them. After the 50% of the features is removed, significant differences between TD and ASD binary masks are found in late time components and in XAI methods, such as, Smooth-Grad Squared, PatternNet, Pattern-Attribution, and LRP.

Metrics

6 Record Views
31 citations in Web of Science
32 citations in Scopus

Details

UN Sustainable Development Goals (SDGs)

This publication has contributed to the advancement of the following goals:

#3 Good Health and Well-Being

InCites Highlights

Data related to this publication, from InCites Benchmarking & Analytics tool:

Collaboration types
Domestic collaboration
International collaboration
Web of Science research areas
Computer Science, Artificial Intelligence
Engineering, Biomedical
Medical Informatics
Logo image