XOR_p A maximally intertwined p-classes problem used as a benchmark with\n built-in truth for neural networks gradient descent optimization

2018/12/18 by Danielle Thierry‐Mieg, Thierry-Mieg, Danielle, Jean Thierry‐Mieg +1
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.1812.07538

openalex publication_date 2018/12/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A natural p-classes generalization of the eXclusive OR problem, the\nsubtraction modulo p, where p is prime, is presented and solved using a single\nfully connected hidden layer with p-neurons. Although the problem is very\nsimple, the landscape is intricate and challenging and represents an\ninteresting benchmark for gradient descent optimization algorithms. Testing 9\noptimizers and 9 activation functions up to p = 191, the method converging most\noften and the fastest to a perfect classification is the Adam optimizer\ncombined with the ELU activation function.\n

Related

• SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem
• PReLU: Yet Another Single-Layer Solution to the XOR Problem
• An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants
• Gradient Descent: The Ultimate Optimizer
• Gradient Descent Fails to Learn High-frequency Functions and Modular Arithmetic