2018/11/28 by Gaurav Sheni, Sheni, Gaurav, Benjamin Schreck +7
Business, Management and Accounting · Computer Science · Mathematics · #Big Data and Business Intelligence #Data Analysis with R #Data Stream Mining Techniques #Data Visualization and Analytics #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Online Learning and Analytics #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1811.11960
openalex publication_date 2018/11/28 · arxiv created 2018/11/29 · arxiv updated 2018/11/30 · openalex created_date 2022/08/01 · openalex updated_date 2026/07/28
In this paper, we present a data science automation system called Prediction Factory. The system uses several key automation algorithms to enable data scientists to rapidly develop predictive models and share them with domain experts. To assess the system's impact, we implemented 3 different interfaces for creating predictive modeling projects: baseline automation, full automation, and optional automation. With a dataset of online grocery shopper behaviors, we divided data scientists among the interfaces to specify prediction problems, learn and evaluate models, and write a report for domain experts to judge whether or not to fund to continue working on. In total, 22 data scientists created 94 reports that were judged 296 times by 26 experts. In a head-to-head trial, reports generated utilizing full data science automation interface reports were funded 57.5% of the time, while the ones that used baseline automation were only funded 42.5% of the time. An intermediate interface which supports optional automation generated reports were funded 58.6% more often compared to the baseline. Full automation and optional automation reports were funded about equally when put head-to-head. These results demonstrate that Prediction Factory has implemented a critical amount of automation to augment the role of data scientists and improve business outcomes.