Files and links (1)
Published, Version of Record (VoR)Open Access Discount via Drexel Libraries Read and Publish Program 2024CC BY V4.0, Open
Journal article
Multilingual Prediction of Cognitive Impairment with Large Language Models and Speech Analysis
Brain sciences, v 14(12), 1292
22 Dec 2024
PMID: 39766491
Featured in Collection : Research Supported by Drexel Libraries' OA Programs
Abstract
Background: Cognitive impairment poses a significant global health challenge, emphasizing the critical need for early detection and intervention. Traditional diagnostics like neuroimaging and clinical evaluations are often subjective, costly, and inaccessible, especially in resource-poor settings. Previous research has focused on speech analysis primarily conducted using English data, leaving multilingual settings unexplored. Methods: In this study, we present our results from the INTERSPEECH 2024 TAUKADIAL Challenge, where we aimed to automatically detect mild cognitive impairment (MCI) and predict cognitive scores for English and Chinese speakers (169 in total). Our approach leverages Whisper, a speech foundation model, to extract language-agnostic speech embeddings. We then utilize ensemble models to incorporate task-specific information. Results: Our model achieved unweighted average recall of 81.83% in an MCI classification task, and root mean squared error of 1.196 in cognitive score prediction task, which placed the model at the second and the first position, respectively, in the ranking for each task. Comparison between language-agnostic and language-specific models reveals the importance of capturing language-specific nuances for accurate cognitive impairment prediction. Conclusions: This study demonstrates the effectiveness of language-specific ensemble modeling with Whisper embeddings in enabling scalable, non-invasive cognitive health assessments of Alzheimer’s disease, achieving state-of-the-art results in multilingual settings.
Metrics
7 Record Views
Details
- Title
- Multilingual Prediction of Cognitive Impairment with Large Language Models and Speech Analysis
- Creators
- Felix Agbavor - Drexel UniversityHualou Liang (Corresponding Author) - Drexel University
- Publication Details
- Brain sciences, v 14(12), 1292
- Publisher
- MDPI
- Number of pages
- 13
- Grant note
- 5P30AG073105 / NIH/NIA
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- School of Biomedical Engineering, Science, and Health Systems
- Web of Science ID
- WOS:001387743800001
- Scopus ID
- 2-s2.0-85213441011
- Other Identifier
- 991022008797404721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Web of Science research areas
- Neurosciences