Utilizing Large Language Models to Generate Synthetic Data to Increase the Performance of BERT-Based Neural Networks

2024/05/08 by Chancellor Woolsey, Woolsey, Chancellor R., P. S. Bisht +5 · 2 citations
Computer Science · Engineering · Medicine · #Advanced Data Processing Techniques #Artificial Intelligence (cs.AI) #COVID-19 diagnosis using AI #Computation and Language (cs.CL) #FOS: Computer and information sciences #Topic Modeling

paper · doi:10.48550/arxiv.2405.06695

openalex publication_date 2024/05/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

An important problem impacting healthcare is the lack of available experts. Machine learning (ML) models may help resolve this by aiding in screening and diagnosing patients. However, creating large, representative datasets to train models is expensive. We evaluated large language models (LLMs) for data creation. Using Autism Spectrum Disorders (ASD), we prompted GPT-3.5 and GPT-4 to generate 4,200 synthetic examples of behaviors to augment existing medical observations. Our goal is to label behaviors corresponding to autism criteria and improve model accuracy with synthetic training data. We used a BERT classifier pretrained on biomedical literature to assess differences in performance between models. A random sample (N=140) from the LLM-generated data was also evaluated by a clinician and found to contain 83% correct behavioral example-label pairs. Augmenting the dataset increased recall by 13% but decreased precision by 16%. Future work will investigate how different synthetic data characteristics affect ML outcomes.

Citations

• Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2020

Cited by

• LLMCARE: early detection of cognitive impairment via transformer models enhanced by LLM-generated synthetic data
• A Scoping Review of Synthetic Data Generation for Biomedical Research and Applications

Related

• Crowdsourcing with Enhanced Data Quality Assurance: An Efficient Approach to Mitigate Resource Scarcity Challenges in Training Large Language Models for Healthcare
• Exploiting ChatGPT for Diagnosing Autism-Associated Language Disorders and Identifying Distinct Features
• Deep learning for autism detection using clinical notes: A comparison of transfer learning for a transparent and black-box approach
• Towards Equitable ASD Diagnostics: A Comparative Study of Machine and Deep Learning Models Using Behavioral and Facial Data
• When Machines Get It Wrong: Large Language Models Perpetuate Autism Myths More Than Humans Do