Test-Time Training with Masked Autoencoders

2022/09/15 by Yossi Gandelsman, Gandelsman, Yossi, Yu Sun +5 · 42 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2209.07522

Project page: https://yossigandelsman.github.io/ttt_mae/index.html

arxiv created 2022/09/15 · arxiv updated 2022/09/16

Abstract

Test-time training adapts to a new test distribution on the fly by optimizing a model for each test input using self-supervision. In this paper, we use masked autoencoders for this one-sample learning problem. Empirically, our simple method improves generalization on many visual benchmarks for distribution shifts. Theoretically, we characterize this improvement in terms of the bias-variance trade-off.

Cited by

• Few-shot Protein Fitness Prediction via In-context Learning and Test-time Training
• Progressive Conditioned Scale-Shift Recalibration of Self-Attention for Online Test-time Adaptation
• Instance-Aware Test-Time Segmentation for Continual Domain Shifts
• Adapting Like Humans: A Metacognitive Agent with Test-time Reasoning
• Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium
• Out-of-Context Misinformation Detection via Variational Domain-Invariant Learning with Test-Time Training
• DiScoFormer: Plug-In Density and Score Estimation with Transformers
• Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models
• Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models
• TTOM: Test-Time Optimization and Memorization for Compositional Video Generation
• End-to-End Test-Time Training for Long Context
• TTRV: Test-Time Reinforcement Learning for Vision Language Models
• ZeroSiam: An Efficient Siamese for Test-Time Entropy Optimization without Collapse
• Online Adaptation via Dual-Stage Alignment and Self-Supervision for Fast-Calibration Brain-Computer Interfaces
• Purge-Gate: Backpropagation-Free Test-Time Adaptation for Point Clouds Classification via Token Purging
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
• Uncertainty-aware Test-Time Training (UT³) for Efficient On-the-fly Domain Adaptive Dense Regression
• ETTA: Efficient Test-Time Adaptation for Vision-Language Models through Dynamic Embedding Updates
• Test-Time Model Adaptation for Quantized Neural Networks
• Test-Time Training for Speech Enhancement
• Low-Cost Test-Time Adaptation for Robust Video Editing
• T³SVFND: Towards an Evolving Fake News Detector for Emergencies with Test-time Training on Short Video Platforms
• CNS-Bench: Benchmarking Image Classifier Robustness Under Continuous Nuisance Shifts
• T3MAL: Test-Time Fast Adaptation for Robust Multi-Scale Information Diffusion Prediction
• Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos
• Synchronizing Task Behavior: Aligning Multiple Tasks during Test-Time Training
• CTA: Cross-Task Alignment for Better Test Time Training
• Context Tuning for In-Context Optimization
• When Test-Time Adaptation Meets Self-Supervised Models
• Orthogonal Projection Subspace to Aggregate Online Prior-knowledge for Continual Test-time Adaptation
• Device-Cloud Collaborative Correction for On-Device Recommendation
• Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation
• Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting
• Test-Time Learning for Large Language Models
• SMART-PC: Skeletal Model Adaptation for Robust Test-Time Training in Point Clouds
• MADCAT: Combating Malware Detection Under Concept Drift with Test-Time Adaptation
• Few-Shot Test-Time Optimization Without Retraining for Semiconductor Recipe Generation and Beyond
• Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging
• SLOT: Sample-specific Language Model Optimization at Test-time
• AdaJEPA: An Adaptive Latent World Model
• Monitor and Recover: A Paradigm for Future Research on Distribution Shift in Learning-Enabled Cyber-Physical Systems
• It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
• BoTTA: Benchmarking on-device Test Time Adaptation
• Self-Bootstrapping for Versatile Test-Time Adaptation

Related

• Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation
• TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision
• Family Matters: A Systematic Study of Spatial vs. Frequency Masking for Continual Test-Time Adaptation
• MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaption