2022/07/25 by Charles Tripp, Jordan Perr‐Sauer, Tripp, Charles Edison +6 · 1 citation
Computer Science · Materials Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Machine Learning in Materials Science #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2207.12547
openalex publication_date 2022/07/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present an empirical dataset surveying the deep learning phenomenon on fully-connected feed-forward multilayer perceptron neural networks. The dataset, which is now freely available online, records the per-epoch training and generalization performance of 483 thousand distinct hyperparameter choices of architectures, tasks, depths, network sizes (number of parameters), learning rates, batch sizes, and regularization penalties. Repeating each experiment an average of 24 times resulted in 11 million total training runs and 40 billion epochs recorded. Accumulating this 1.7 TB dataset utilized 11 thousand CPU core-years, 72.3 GPU-years, and 163 node-years. In surveying the dataset, we observe durable patterns persisting across tasks and topologies. We aim to spark scientific study of machine learning techniques as a catalyst for the theoretical discoveries needed to progress the field beyond energy-intensive and heuristic practices.