2019/07/04 by Robert Bamler, Bamler, Robert, Stephan Mandt +1
Computer Science · Physics and Astronomy · #Computability, Logic, AI Algorithms #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Statistical Mechanics (cond-mat.stat-mech) #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.1907.02163
openalex publication_date 2019/07/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Continuous symmetries and their breaking play a prominent role in contemporary physics. Effective low-energy field theories around symmetry breaking states explain diverse phenomena such as superconductivity, magnetism, and the mass of nucleons. We show that such field theories can also be a useful tool in machine learning, in particular for loss functions with continuous symmetries that are spontaneously broken by random initializations. In this paper, we illuminate our earlier published work (Bamler & Mandt, 2018) on this topic more from the perspective of theoretical physics. We show that the analogies between superconductivity and symmetry breaking in temporal representation learning are rather deep, allowing us to formulate a gauge theory of `charged' embedding vectors in time series models. We show that making the loss function gauge invariant speeds up convergence in such models.