2024/05/30 by Minh Nguyen, Minh‐Hoang Nguyen, Nguyen, Minh +10 · 1 voice · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Face recognition and analysis #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #cs.LG
paper · pdf · doi:10.48550/arxiv.2405.20448
openalex publication_date 2024/05/30 · arxiv published 2024/05/30 · arxiv updated 2025/07/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Deep learning models benefit from rich (e.g., multi-modal) input features. However, multimodal models might be challenging to deploy, because some inputs may be missing at inference. Current popular solutions include marginalization, imputation, and training multiple models. Marginalization achieves calibrated predictions, but it is computationally expensive and only feasible for low dimensional inputs. Imputation may result in inaccurate predictions, particularly when high-dimensional data, such as images, are missing. Training multiple models, where each model is designed to handle different subsets of inputs, can work well but requires prior knowledge of missing input patterns. Furthermore, training and retaining multiple models can be costly. We propose an efficient method to learn both the conditional distribution using full inputs and the marginal distributions. Our method, Knockout, randomly replaces input features with appropriate placeholder values during training. We provide a theoretical justification for Knockout and show that it can be interpreted as an implicit marginalization strategy. We evaluate Knockout across a wide range of simulations and real-world datasets and show that it offers strong empirical performance.