Susan Manzi

Background/Purpose: Antinuclear antibody (ANA) immunofluorescence (IFA) patterns and titres are a key part of rheumatology diagnostics, however, there is considerable intra- and interlaboratory variability with manual interpretation. Replacing manual interpretation with a standardized and automated approach could help reduce variability, increasing laboratory accuracy and efficiency. We developed machine learning (ML) models (ANA Reader©) to aid laboratories in ANA pattern and titre interpretation, including a model for the nuclear dense fine-speckled (DFS) ANA pattern (AC-2), a rare pattern among systemic autoimmune rheumatic disease (SARD) patients that decreases the likelihood of these conditions.
Methods: 13,671 ANA images from SLE patients enrolled in the Systemic Lupus International Collaborating Clinics Inception Cohort (SLICC, n=2,825 images), non-SLE subjects enrolled in the Ontario Health Study (OHS, n=10,639 images), and the International Consensus on ANA Patterns (ICAP, n=207 images) were analyzed. All SLICC and OHS ANA were performed in 1 central laboratory using IFA on HEp-2 cells (NovaLite, Werfen, SD) and read on a digital IFA microscope (NovaView, Werfen, SD). As the reference standard, 1 laboratory technologist (HH) with >30 years of experience in ANA studies interpreted 13 ANA patterns and titre for each image. We developed and compared the performance of 8 ML models for ANA pattern recognition. To evaluate ANA titre, we used an ML technique for imaging processing that identified individual HEp-2 cells in the ANA images and then calculated the cell illuminance and cut-offs corresponding to each titre (1:80-1:5120). Fifty images were randomly selected to compare the titre classification based on image processing with the lab technologist as the reference standard.
Results: 6,307 images containing at least 1 ANA pattern (≥1:80) from SLICC (n=2,806 images), OHS (n=3339 images), and ICAP (n=162 images) were included. We identified 1 ML model (ANA Reader©) with the best performance for ANA pattern identification compared to the reference with a high area under the curve (AUC) score of 83.4%, modest weighted accuracy of 68.4%, precision of 67.1%, sensitivity of 70.1%, and F1 score of 67.2%. The AUC for individual ANA patterns ranged from 0.71 to 0.97 (Figure 1). There was a strong correlation between titres reported by the ANA Reader© and the technologist’s interpretation (Spearman rank 0.93, p <0.0001), where the titres reported were identical or differed Figure 1. Area under the curve (AUC) scores for the 13 antinuclear antibody (ANA) patterns using the ANA Reader© model, which had the best performance compared to 7 other machine learning techniques. by only 1 dilution in most cases (96.0%). The ANA patterns with the best performance were centromere (AUC 0.97) and pleomorphic patterns (AUC 0.97). On average, there were 5 images per patient sample for SLICC, 3 images per patient sample for OHS, and 1 image per patient from ICAP. 80% of the images were used for model training and the remaining 20% for validation. In total, there were 512 patients in the SLICC cohort, 3,559 individuals in the OHS cohort, and 207 patients from ICAP who were included in the study. Conclusions: ML has the potential to become a highly effective and efficient approach to evaluating ANA patterns and titres. The performance of our ANA Reader© is expected to improve as we continue to train our models with more ANA images. Future external validation studies and the development of other ML models to predict more complex and multiple ANA patterns and titres are also underway.