Semi-Supervised Learning with Declaratively Specified Entropy Constraints

2018/04/24 by Haitian Sun, William W. Cohen, Sun, Haitian +3
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1804.09238

openalex publication_date 2018/04/24 · openalex created_date 2018/05/07 · openalex updated_date 2026/07/28

Abstract

We propose a technique for declaratively specifying strategies for semi-supervised learning (SSL). The proposed method can be used to specify ensembles of semi-supervised learning, as well as agreement constraints and entropic regularization constraints between these learners, and can be used to model both well-known heuristics such as co-training and novel domain-specific heuristics. In addition to representing individual SSL heuristics, we show that multiple heuristics can also be automatically combined using Bayesian optimization methods. We show consistent improvements on a suite of well-studied SSL benchmarks, including a new state-of-the-art result on a difficult relation extraction task.

Citations

• Revisiting Semi-Supervised Learning with Graph Embeddings
• Bootstrapping Distantly Supervised IE using Joint Learning and Small Well-structured Corpora
• Semi-Supervised Classification with Graph Convolutional Networks

Related

• Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning
• A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning
• Improving Pre-trained Self-Supervised Embeddings Through Effective Entropy Maximization
• Can semi-supervised learning use all the data effectively? A lower bound perspective
• An Information-theoretical Approach to Semi-supervised Learning under Covariate-shift