Robust unsupervised domain adaptation for neural networks via moment alignment

2017/11/30 by Werner Zellinger, Bernhard A. Moser, Bernhard Moser +5 · 97 citations
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Artificial neural network #Computer science #Domain Adaptation and Few-Shot Learning #Feature vector #Machine Learning and ELM #Mathematics #Metric (unit) #Pattern recognition (psychology) #Regularization (linguistics) #Speech Recognition and Synthesis #cs.LG #stat.ML

paper · pdf · doi:10.1016/j.ins.2019.01.025

published in Information Sciences 483, 174-191 (Elsevier BV) · Preliminary version of this work appeared in ICLR

openalex created_date 2017/12/04 · openalex publication_date 2019/01/10 · arxiv created 2019/08/13 · arxiv updated 2019/08/14 · openalex updated_date 2026/08/05

Abstract

A novel approach for unsupervised domain adaptation for neural networks is proposed. It relies on metric-based regularization of the learning process. The metric-based regularization aims at domain-invariant latent feature representations by means of maximizing the similarity between domain-specific activation distributions. The proposed metric results from modifying an integral probability metric such that it becomes less translation-sensitive on a polynomial function space. The metric has an intuitive interpretation in the dual space as the sum of differences of higher order central moments of the corresponding activation distributions. Under appropriate assumptions on the input distributions, error minimization is proven for the continuous case. As demonstrated by an analysis of standard benchmark experiments for sentiment analysis, object recognition and digit recognition, the outlined approach is robust regarding parameter changes and achieves higher classification accuracies than comparable approaches. The source code is available at https://github.com/wzell/mann.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Adam: A Method for Stochastic Optimization
• Deep Domain Confusion: Maximizing for Domain Invariance
• Domain Adaptation: Learning Bounds and Algorithms
• The Variational Fair Autoencoder
• Domain adaptation and sample bias correction theory and algorithm for regression
• A note on the summation of Chebyshev series
• Marginalized Denoising Autoencoders for Domain Adaptation
• Learning Transferable Features with Deep Adaptation Networks
• ImageNet classification with deep convolutional neural networks
• Representation Learning: A Review and New Perspectives
• A Survey on Transfer Learning
• Unsupervised Domain Adaptation with Residual Transfer Networks
• Revisiting Batch Normalization For Practical Domain Adaptation
• Integral Probability Metrics and Their Generating Classes of Functions
• Domain Separation Networks
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Deep learning