Logo image
Back
Applying bootstrap statistics to assess the validity of utilizing fully synthetic patients in AI machine learning algorithms
Journal article   Peer reviewed

Applying bootstrap statistics to assess the validity of utilizing fully synthetic patients in AI machine learning algorithms

John Nakayama, Mike McGaughey, Tiffany Summerscales, Sarina Ortiz, Grace Pindzola, Teresa Hong, Eirwen Miller, Christopher Morse, Thomas Krivak, Sarah Crafton, …
Gynecologic oncology, v 208, pp S430-S430
May 2026
DOI: https://doi.org/10.1016/j.ygyno.2026.01.737

Abstract

Objectives Expanding on our previous work predicting progression-free survival in ovarian cancer using augmented datasets (synthetic plus real patient data), this study sought to evaluate the performance and validity of machine learning models trained solely on AI-generated synthetic patient data to predict platinum sensitivity. Methods A cohort of 547 patients with all stages of epithelial ovarian cancer was identified from January 2017 to October 2023. AI training included the following data points known at the end of adjuvant chemotherapy: age, stage, medical comorbidities, histology, grade, CA125, BRCA status, frontline chemotherapy regimen, and planned maintenance therapy (bevacizumab or PARP inhibitor). A Variational Autoencoder (VAE) was trained on this real-world dataset to generate 1000 synthetic datasets, each containing 547 synthetic patients. Hyperparameter tuning and an increased variable count enhanced model accuracy. Random Forest, logistic regression, and Support Vector Machine) algorithms were then trained and tested on only the synthetic datasets to predict platinum sensitivity. Lastly, the performance of these 3 models was evaluated by assessing their ability to accurately predict platinum sensitivity on a second, independent real-world data set. We employed bootstrap statistics to rigorously evaluate the performance and validity of this synthetic data-driven approach. Accuracy and F1-score were recorded across 1000 iterations, generating performance distributions for each algorithm. Results When models were trained and tested using 1000 synthetic datasets, performance(prediction of platinum sensitivity) varied by model type. The Support Vector Machine (SVM) model achieved the highest mean accuracy of 70.1% (SD = 2.9%) with a mean F1 score of 66.6% (SD = 3.5%). Logistic regression achieved a mean accuracy of 67.6% (SD = 3.5%) and a mean F1 score of 65.9% (SD = 3.3%). The Random Forest model yielded a mean accuracy of 67.4% (SD = 2.9%) and a mean F1 score of 65.4% (SD = 3.0%). The mean accuracy of a model trained with the real-world data set was 81.8% and the mean F1 score was 81.3%. Conclusions While models trained on real-world data currently demonstrate superior predictive performance, our findings suggest that synthetic data offers a viable alternative, particularly in scenarios where data sharing is not feasible. Synthetic data may enable collaboration without compromising patient privacy, protect competitively sensitive information, and address data governance limitations. Future research should focus on optimizing synthetic data generation techniques and refining model selection strategies to further enhance the accuracy, reliability, and clinical utility of predictions derived solely from synthetic datasets.

Metrics

1 Record Views

Details

Logo image