Instance-Wise Adaptive Sampling for Dataset Construction in Approximating Inverse Problem Solutions

2025/09/04 by Jiequn Han, Kui Ren, Han, Jiequn +3
Computer Science · Mathematics · #Advanced Multi-Objective Optimization Algorithms #Computational Physics (physics.comp-ph) #FOS: Computer and information sciences #FOS: Mathematics #FOS: Physical sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Numerical Analysis (math.NA) #Numerical methods in inverse problems

paper · pdf · doi:10.48550/arxiv.2509.04583

openalex publication_date 2025/09/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose an instance-wise adaptive sampling framework for constructing compact and informative training datasets for supervised learning of inverse problem solutions. Typical learning-based approaches aim to learn a general-purpose inverse map from datasets drawn from a prior distribution, with the training process independent of the specific test instance. When the prior has a high intrinsic dimension or when high accuracy of the learned solution is required, a large number of training samples may be needed, resulting in substantial data collection costs. In contrast, our method dynamically allocates sampling effort based on the specific test instance, enabling significant gains in sample efficiency. By iteratively refining the training dataset conditioned on the latest prediction, the proposed strategy tailors the dataset to the geometry of the inverse map around each test instance. We demonstrate the effectiveness of our approach in the inverse scattering problem under two types of structured priors. Our results show that the advantage of the adaptive method becomes more pronounced in settings with more complex priors or higher accuracy requirements. While our experiments focus on a particular inverse problem, the adaptive sampling strategy is broadly applicable and readily extends to other inverse problems, offering a scalable and practical alternative to conventional fixed-dataset training regimes.

Citations

• Multi-fidelity Parameter Estimation Using Conditional Diffusion Models
• Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
• The Limits of Inference Scaling Through Resampling
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• Back-Projection Diffusion: Solving the Wideband Inverse Scattering Problem with Diffusion Models
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• Recursive Introspection: Teaching Language Model Agents How to Self-Improve
• A Direct Sampling Method and Its Integration with Deep Learning for Inverse Scattering Problems with Phaseless Data
• Reconstructing the shape and material parameters of dissipative obstacles using an impedance model
• Scaling Data-Constrained Language Models
• A Direct Sampling-Based Deep Learning Approach for Inverse Medium Scattering Problems
• Self-Refine: Iterative Refinement with Self-Feedback
• Generating Sequences by Learning to Self-Correct
• Diffusion Posterior Sampling for General Noisy Inverse Problems
• Large deviation theory-based adaptive importance sampling for rare events in high dimensions
• Reinforced Inverse Scattering
• Coupling Deep Learning with Full Waveform Inversion
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Probabilistic Models
• Deep Learning Techniques for Inverse Problems in Imaging
• DeepXDE: A Deep Learning Library for Solving Differential Equations
• High resolution inverse scattering in two dimensions using recursive linearization
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Related

• Reinforcement Learning of Adaptive Acquisition Policies for Inverse Problems
• Learned reconstruction methods for inverse problems: sample error estimates
• Learned Regularization for Inverse Problems: Insights from a Spectral Model
• Accelerating Inverse Learning via Intelligent Localization with Exploratory Sampling
• On Learning the Optimal Regularization Parameter in Inverse Problems