HARK Side of Deep Learning -- From Grad Student Descent to Automated\n Machine Learning

2019/04/16 by Oguzhan Gencoglu, Mark van Gils, Gencoglu, Oguzhan +13 · 3 citations
Business, Management and Accounting · Computer Science · Social Sciences · #Adversarial Robustness in Machine Learning #Big Data and Business Intelligence #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1904.07633

openalex publication_date 2019/04/16 · openalex created_date 2022/07/29 · openalex updated_date 2026/07/28

Abstract

Recent advancements in machine learning research, i.e., deep learning,\nintroduced methods that excel conventional algorithms as well as humans in\nseveral complex tasks, ranging from detection of objects in images and speech\nrecognition to playing difficult strategic games. However, the current\nmethodology of machine learning research and consequently, implementations of\nthe real-world applications of such algorithms, seems to have a recurring\nHARKing (Hypothesizing After the Results are Known) issue. In this work, we\nelaborate on the algorithmic, economic and social reasons and consequences of\nthis phenomenon. We present examples from current common practices of\nconducting machine learning research (e.g. avoidance of reporting negative\nresults) and failure of generalization ability of the proposed algorithms and\ndatasets in actual real-life usage. Furthermore, a potential future trajectory\nof machine learning research and development from the perspective of\naccountable, unbiased, ethical and privacy-aware algorithmic decision making is\ndiscussed. We would like to emphasize that with this discussion we neither\nclaim to provide an exhaustive argumentation nor blame any specific institution\nor individual on the raised issues. This is simply a discussion put forth by\nus, insiders of the machine learning field, reflecting on us.\n

Citations

• On Large-Batch Training for Deep Learning: Generalization Gap and Sharp\n Minima
• Neural Architecture Search with Reinforcement Learning
• Neural Optimizer Search with Reinforcement Learning
• Counterfactual Fairness
• Interpretation of Neural Networks is Fragile
• Progressive Neural Architecture Search
• Efficient Neural Architecture Search via Parameter Sharing
• AutoAugment: Learning Augmentation Policies from Data
• Do CIFAR-10 Classifiers Generalize to CIFAR-10?
• Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
• Expanding search in the space of empirical ML
• A Genetic Programming Approach to Designing Convolutional Neural Network Architectures
• Do ImageNet Classifiers Generalize to ImageNet?

Cited by

• Not All Explanations for Deep Learning Phenomena Are Equally Valuable
• Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural\n Networks and Neural Architecture Search
• NAS evaluation is frustratingly hard

Related

• Trustworthy Machine Learning under Social and Adversarial Data Sources
• The reanimation of pseudoscience in machine learning and its ethical repercussions
• The Return of Pseudosciences in Artificial Intelligence: Have Machine Learning and Deep Learning Forgotten Lessons from Statistics and History?
• On the Societal Impact of Machine Learning
• (Machine) Learning to Be Like Thee? For Algorithm Education, Not Training