2017/07/23 by Michael C. Hughes, Leah Weiner, Hughes, Michael C. +11 · 1 citation
Computer Science · Social Sciences · #Topic Modeling #Machine Learning in Healthcare #Computational and Text Analysis Methods
paper · pdf · doi:10.48550/arxiv.1707.07341
Supervisory signals have the potential to make low-dimensional data\nrepresentations, like those learned by mixture and topic models, more\ninterpretable and useful. We propose a framework for training latent variable\nmodels that explicitly balances two goals: recovery of faithful generative\nexplanations of high-dimensional data, and accurate prediction of associated\nsemantic labels. Existing approaches fail to achieve these goals due to an\nincomplete treatment of a fundamental asymmetry: the intended application is\nalways predicting labels from data, not data from labels. Our\nprediction-constrained objective for training generative models coherently\nintegrates loss-based supervisory signals while enabling effective\nsemi-supervised learning from partially labeled data. We derive learning\nalgorithms for semi-supervised mixture and topic models using stochastic\ngradient descent with automatic differentiation. We demonstrate improved\nprediction quality compared to several previous supervised topic models,\nachieving predictions competitive with high-dimensional logistic regression on\ntext sentiment analysis and electronic health records tasks while\nsimultaneously learning interpretable topics.\n