Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation

2025/10/10 by Xiaofeng Cao, Cao, Xiaofeng, Mingwei Xu +19 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2510.08962

openalex publication_date 2025/10/10 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Learning with high-resource data has demonstrated substantial success in artificial intelligence (AI); however, the costs associated with data annotation and model training remain significant. A fundamental objective of AI research is to achieve robust generalization with limited-resource data. This survey employs agnostic active sampling theory within the Probably Approximately Correct (PAC) framework to analyze the generalization error and label complexity associated with learning from low-resource data in both model-agnostic supervised and unsupervised settings. Based on this analysis, we investigate a suite of optimization strategies tailored for low-resource data learning, including gradient-informed optimization, meta-iteration optimization, geometry-aware optimization, and LLMs-powered optimization. Furthermore, we provide a comprehensive overview of multiple learning paradigms that can benefit from low-resource data, including domain transfer, reinforcement feedback, and hierarchical structure modeling. Finally, we conclude our analysis and investigation by summarizing the key findings and highlighting their implications for learning with low-resource data.

Citations

• Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Data
• MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration
• Single Domain Generalization for Few-Shot Counting via Universal Representation Matching
• Is a Good Foundation Necessary for Efficient Reinforcement Learning? The Computational Role of the Base Model in Exploration
• Efficient Online Reinforcement Learning for Diffusion Policy
• VARFVV: View-Adaptive Real-Time Interactive Free-View Video Streaming with Edge Computing
• The impact of intrinsic rewards on exploration in Reinforcement Learning
• Each Graph is a New Language: Graph Learning with LLMs
• Consent in Crisis: The Rapid Decline of the AI Data Commons
• SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning
• On the Feasibility of Fidelity^- for Graph Pruning
• LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuning
• Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximation
• The Importance of Directional Feedback for LLM-based Optimizers
• LLM4ED: Large Language Models for Automatic Equation Discovery
• UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation
• Hierarchical Decision Making Based on Structural Information Principles
• Domain Generalization through Meta-Learning: A Survey
• Large Language Model-Based Evolutionary Optimizer: Reasoning with elitism
• Large Language Models As Evolution Strategies
• Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers
• Determinants of LLM-assisted Decision-Making
• LLaGA: Large Language and Graph Assistant
• Large Language Model Meets Graph Neural Network in Knowledge Distillation
• ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution
• StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving
• GraphGPT: Graph Instruction Tuning for Large Language Models
• Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs
• Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learning
• Automatic Prompt Optimization with "Gradient Descent" and Beam Search
• Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference
• Meta-causal Learning for Single Domain Generalization
• Contrastive Alignment of Vision to Language Through Parameter-Efficient Transfer Learning
• LLaMA: Open and Efficient Foundation Language Models
• UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling
• Domain Re-Modulation for Few-Shot Generative Domain Adaptation
• Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling
• Graph Few-shot Learning with Task-specific Structures
• Cell-Free Latent Go-Explore
• Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets
• Data-Efficient Learning via Minimizing Hyperspherical Energy
• LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning
• Few-Shot Adaptation of Pre-Trained Networks for Domain Shift
• A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities
• Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment
• Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty
• Sample Efficient Deep Reinforcement Learning via Uncertainty Estimation
• Meta Propagation Networks for Graph Few-shot Semi-supervised Learning
• VL-Adapter: Parameter-Efficient Transfer Learning for\n Vision-and-Language Tasks
• Mastering Atari Games with Limited Data
• Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation
• Towards a Unified View of Parameter-Efficient Transfer Learning
• Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencoder
• Few-Shot Domain Adaptation For End-to-End Communication
• Cross-domain Few-shot Learning with Task-specific Adapters
• Model-Based Reinforcement Learning via Latent-Space Collocation
• TOHAN: A One-step Approach towards Few-shot Hypothesis Adaptation
• Provably Faster Algorithms for Bilevel Optimization
• ECKPN: Explicit Class Knowledge Propagation Network for Transductive Few-shot Learning
• The Power of Scale for Parameter-Efficient Prompt Tuning
• Few-shot Image Generation via Cross-domain Correspondence
• Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation
• Progressive Domain Expansion Network for Single Domain Generalization
• A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum
• Decentralized Riemannian Gradient Descent on the Stiefel Manifold
• A Unified Lottery Ticket Hypothesis for Graph Neural Networks
• Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs
• Hierarchical Reinforcement Learning By Discovering Intrinsic Options
• Hyperbolic Deep Neural Networks: A Survey
• Parameter-Efficient Transfer Learning with Diff Pruning
• On Self-Distilling Graph Neural Network
• Behavior Priors for Efficient Reinforcement Learning
• A Survey on Recent Approaches for Natural Language Processing in\n Low-Resource Scenarios
• Bilevel Optimization: Convergence Analysis and Enhanced Design
• Mastering Atari with Discrete World Models
• A Survey of Deep Active Learning
• Offline Meta-Reinforcement Learning with Advantage Weighting
• Node Classification on Graphs with Few-Shot Novel Labels via Meta Transformed Network Embedding
• Few-shot Relation Extraction via Bayesian Meta-learning on Relation Graphs
• On the Global Optimality of Model-Agnostic Meta-Learning
• Graph Prototypical Networks for Few-shot Learning on Attributed Networks
• Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning
• Language Models are Few-Shot Learners
• MOPO: Model-based Offline Policy Optimization
• Generalized Reinforcement Meta Learning for Few-Shot Optimization
• Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation
• Meta-Learning in Neural Networks: A Survey
• DPGN: Distribution Propagation Graph Network for Few-shot Learning
• Learning to Learn Single Domain Generalization
• Distilling Knowledge from Graph Convolutional Networks
• Meta-learning curiosity algorithms
• Differentiating through the Fréchet Mean
• RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments
• Adaptive Region-Based Active Learning
• Never Give Up: Learning Directed Exploration Strategies
• Few-shot Domain Adaptation by Causal Mechanism Transfer
• TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning
• Dream to Control: Learning Behaviors by Latent Imagination
• Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation
• Hyperbolic Graph Neural Networks
• Hyperbolic Graph Convolutional Neural Networks
• Graph Few-shot Learning via Knowledge Transfer
• Learning Temporal Action Proposals With Fewer Labels
• Meta-Learning with Implicit Gradients
• GraphSAINT: Graph Sampling Based Inductive Learning Method
• When to Trust Your Model: Model-Based Policy Optimization
• Domain-Specific Batch Normalization for Unsupervised Domain Adaptation
• Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot\n Learning on Category Graph
• Edge-labeling Graph Neural Network for Few-shot Learning
• Generating Classification Weights with GNN Denoising Autoencoders for\n Few-Shot Learning
• f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
• Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic\n Context Variables
• High-Fidelity Image Generation With Fewer Labels
• Simplifying Graph Convolutional Networks
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Partial Label Learning with Self-Guided Retraining
• Parameter-Efficient Transfer Learning for NLP
• Go-Explore: a New Approach for Hard-Exploration Problems
• Label Efficient Semi-Supervised Learning via Graph Filtering
• Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions
• Meta-Transfer Learning for Few-Shot Learning
• Learning Latent Dynamics for Planning from Pixels
• Exploration by Random Network Distillation
• Episodic Curiosity through Reachability
• Network centrality: an introduction
• Understanding Batch Normalization
• Sample-Efficient Deep Reinforcement Learning via Episodic Backward Update
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
• Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
• Data-Efficient Hierarchical Reinforcement Learning
• Task-Agnostic Meta-Learning for Few-shot Learning
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Diversity is All You Need: Learning Skills without a Reward Function
• FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling
• Domain Generalization by Marginal Transfer Learning
• Learning to Compare: Relation Network for Few-Shot Learning
• Few-Shot Learning with Graph Neural Networks
• Riemannian Stein Variational Gradient Descent for Bayesian Inference
• Few-Shot Adversarial Domain Adaptation
• Learning to Generalize: Meta-Learning for Domain Generalization
• Representation Learning on Graphs: Methods and Applications
• Hindsight Experience Replay
• Attention Is All You Need
• Inductive Representation Learning on Large Graphs
• Poincaré Embeddings for Learning Hierarchical Representations
• Curiosity-driven Exploration by Self-supervised Prediction
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Count-Based Exploration with Neural Density Models
• An overview of gradient descent optimization algorithms
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference\n Algorithm
• Learning feed-forward one-shot learners
• Matching Networks for One Shot Learning
• Unifying Count-Based Exploration and Intrinsic Motivation
• A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation
• A Kernel Test of Goodness of Fit
• Variational Inference: A Review for Statisticians
• Prioritized Experience Replay
• Measuring Sample Quality with Stein's Method
• Variational Inference with Normalizing Flows
• Kernel Mean Shrinkage Estimators
• How Does the Brain Solve Visual Object Recognition?
• Importance Weighted Active Learning
• Deep Blue
• EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers

Cited by

• Preference-driven Knowledge Distillation for Few-shot Node Classification
• MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restoration
• Dual-granularity Sinkhorn Distillation for Enhanced Learning from Long-tailed Noisy Data

Related

• A Survey of Learning on Small Data: Generalization, Optimization, and Challenge
• Learned Optimizers that Scale and Generalize
• Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws
• From Weakly Supervised Learning to Active Learning
• Learning from Limited and Imperfect Data