Exploring the Role of Machine Learning in Scientific Workflows:\n Opportunities and Challenges

2021/10/26 by Azita Nouri, Nouri, Azita, Philip E. Davis +5
Business, Management and Accounting · Computer Science · Decision Sciences · #Big Data and Business Intelligence #Data Stream Mining Techniques #Distributed #FOS: Computer and information sciences #Parallel #Research Data Management Practices #Scientific Computing and Data Management #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2110.13999

openalex publication_date 2021/10/26 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

In this survey, we discuss the challenges of executing scientific workflows\nas well as existing Machine Learning (ML) techniques to alleviate those\nchallenges. We provide the context and motivation for applying ML to each step\nof the execution of these workflows. Furthermore, we provide recommendations on\nhow to extend ML techniques to unresolved challenges in the execution of\nscientific workflows. Moreover, we discuss the possibility of using ML\ntechniques for in-situ operations. We explore the challenges of in-situ\nworkflows and provide suggestions for improving the performance of their\nexecution using ML techniques.\n

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• NP-complete scheduling problems
• Big Data, new epistemologies and paradigm shifts
• End to End Learning for Self-Driving Cars

Related

• Machine Learning-Driven Predictive Resource Management in Complex Science Workflows
• Machine Learning Operations: A Mapping Study
• Applications and Techniques for Fast Machine Learning in Science
• Machine Learning Operations (MLOps): Overview, Definition, and Architecture
• Machine learning operations (mlops) : Overview, definition, and architecture