2017/03/16 by David Belanger, Belanger, David, Bishan Yang +3 · 31 citations
Computer Science · Mathematics · #Artificial intelligence #Computer science #Convex function #Deep learning #Domain Adaptation and Few-Shot Learning #End-to-end principle #Energy (signal processing) #Energy minimization #FOS: Computer and information sciences #Function (biology) #Generative Adversarial Networks and Image Synthesis #Image (mathematics) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Mathematics #Minification #Physics #Regular polygon #Statistics #Structured prediction #Topic Modeling #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1703.05667
published in arXiv (Cornell University), 429-439 (Cornell University) · ICML 2017
openalex publication_date 2017/03/16 · arxiv created 2017/07/15 · arxiv updated 2017/07/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Structured Prediction Energy Networks (SPENs) are a simple, yet expressive family of structured prediction models (Belanger and McCallum, 2016). An energy function over candidate structured outputs is given by a deep network, and predictions are formed by gradient-based optimization. This paper presents end-to-end learning for SPENs, where the energy function is discriminatively trained by back-propagating through gradient-based prediction. In our experience, the approach is substantially more accurate than the structured SVM method of Belanger and McCallum (2016), as it allows us to use more sophisticated non-convex energies. We provide a collection of techniques for improving the speed, accuracy, and memory requirements of end-to-end SPENs, and demonstrate the power of our method on 7-Scenes image denoising and CoNLL-2005 semantic role labeling tasks. In both, inexact minimization of non-convex SPEN energies is superior to baseline methods that use simplistic energy functions that can be minimized exactly.