Synthetic Data for Deep Learning

2019/09/25 by Sergey Nikolenko, Sergey I. Nikolenko, Nikolenko, Sergey I. · 1 voice · 15 citations
Computer Science · #Neural Networks and Applications #Time Series Analysis and Forecasting #cs.CR #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.1909.11512

openalex publication_date 2019/09/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Synthetic data is an increasingly popular tool for training deep learning models, especially in computer vision but also in other areas. In this work, we attempt to provide a comprehensive survey of the various directions in the development and application of synthetic data. First, we discuss synthetic datasets for basic computer vision problems, both low-level (e.g., optical flow estimation) and high-level (e.g., semantic segmentation), synthetic environments and datasets for outdoor and urban scenes (autonomous driving), indoor scenes (indoor navigation), aerial navigation, simulation environments for robotics, applications of synthetic data outside computer vision (in neural programming, bioinformatics, NLP, and more); we also survey the work on improving synthetic data development and alternative ways to produce it such as GANs. Second, we discuss in detail the synthetic-to-real domain adaptation problem that inevitably arises in applications of synthetic data, including synthetic-to-real refinement with GAN-based models and domain adaptation at the feature/model level without explicit data transformations. Third, we turn to privacy-related applications of synthetic data and review the work on generating synthetic datasets with differential privacy guarantees. We conclude by highlighting the most promising directions for further work in synthetic data studies.

Cited by

• Learning Structural Convergence: A Neuro-Symbolic Benchmark for Temporal Reasoning
• Digital phantoms in medical research: Synthetic data and the pursuit of ground truth
• Finer-Personalization Rank: Fine-Grained Retrieval Examines Identity Preservation for Personalized Generation
• Measuring the Effect of Background on Classification and Feature Importance in Deep Learning for AV Perception
• FOD-S2R: A FOD Dataset for Sim2Real Transfer Learning based Object Detection
• TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding
• Canonical Space Representation for 4D Panoptic Segmentation of Articulated Objects
• On the Conceptualization of Synthetic Data: Why Not All Synthetic Data are Useful and Some are
• Detection of Coastal Flooding With TinyCamML: A Low‐Cost, Privacy‐Preserving Cellular‐Connected Camera With Onboard ML
• Synthetic-to-Real Transfer Learning for Chromatin-Sensitive PWS Microscopy
• Synthetic data as meaningful data. On Responsibility in data ecosystems
• Fake & Square: Training Self-Supervised Vision Transformers with Synthetic Data and Synthetic Hard Negatives
• Case Studies of Generative Machine Learning Models for Dynamical Systems
• DALI-PD: Diffusion-based Synthetic Layout Heatmap Generation for ML in Physical Design
• Development of Hybrid Artificial Intelligence Training on Real and Synthetic Data: Benchmark on Two Mixed Training Strategies

Discussions

• Synthetic Data for Deep Learning (2019) [hn, 95 points, 48 comments]

Related

• Synthetic Image Data for Deep Learning
• Domain Adaptation of Synthetic Driving Datasets for Real-World Autonomous Driving
• How much real data do we actually need: Analyzing object detection performance using synthetic and real data
• Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications
• Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization