Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision

2021/09/16 by David Picard, Picard, David · 8 voices · 11 citations
#cs.CV

paper · pdf · doi:10.48550/arxiv.2109.08203

Abstract

In this paper I investigate the effect of random seed selection on the accuracy when using popular deep learning architectures for computer vision. I scan a large amount of seeds (up to 10⁴) on CIFAR 10 and I also scan fewer seeds on Imagenet using pre-trained models to investigate large scale datasets. The conclusions are that even if the variance is not very large, it is surprisingly easy to find an outlier that performs much better or much worse than the average.

Cited by

• The Hitchhiker's Guide to Monoculture
• Parameter efficient hybrid spiking-quantum convolutional neural network with surrogate gradient and quantum data-reupload
• Parameter-Free Clustering via Self-Supervised Consensus Maximization (Extended Version)
• The influence of the random numbers quality on the results in stochastic simulations and machine learning
• Data as a Lever: A Neighbouring Datasets Perspective on Predictive Multiplicity
• Architecture Is All You Need: Diversity-Enabled Sweet Spots for Robust Humanoid Locomotion
• Hybrid Layer-Wise ANN-SNN With Surrogate Spike Encoding-Decoding Structure
• Accuracy-Robustness Trade Off via Spiking Neural Network Gradient Sparsity Trail
• Explicit modelling of subject dependency in BCI decoding
• Algorithmic Tradeoffs, Applied NLP, and the State-of-the-Art Fallacy
• Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)

Discussions

• Not LLM, but shameless plug nonetheless: arxiv.org/abs/2109.08203 [bsky, 9 points, 1 comments]
• Disregard that last order, tune your seed. arxiv.org/abs/2109.08203 [bsky, 8 points, 0 comments]
• This work was partly inspired by the amazing torch.manual_seed(3407) is all you need arxiv.org/abs/2109.08203 by my PhD advisor @davidpicard.eurosky.social It recently resurfaced with @karpathy.bsky.s [bsky, 3 points, 1 comments]
• Torch.manual_seed(3407) is all you need: On the influence of random seeds [hn, 2 points, 0 comments]
• The influence of random seeds in deep learning architectures for computer vision [hn, 1 points, 0 comments]
• Or is it! arxiv.org/abs/2109.08203 [bsky, 1 points, 0 comments]
• eed = 42, it's the number derived as the ultimate answer in the SF novel "The Hitchhiker's Guide to the Galaxy," so it's a joke. I occasionally see seed = 3407, but I didn't know the reason for it. [bsky, 0 points, 0 comments]
• arxiv.org/abs/2109.08203 [bsky, 0 points, 0 comments]

Related

• Statistical Foundation Behind Machine Learning and Its Impact on Computer Vision
• Ground-truth or DAER: Selective Re-query of Secondary Information
• Robust Computer Vision in an Ever-Changing World: A Survey of Techniques for Tackling Distribution Shifts
• Finding and Removing Clever Hans: Using Explanation Methods to Debug and\n Improve Deep Models
• Randomness In Neural Network Training: Characterizing The Impact of Tooling