2019/03/12 by Karl Mason, Mason, Karl, Sadegh Vejdan +3
Business, Management and Accounting · Decision Sciences · Computer Science · #Big Data and Business Intelligence #Big Data Technologies and Applications #Privacy-Preserving Technologies in Data
paper · pdf · doi:10.48550/arxiv.1903.06798
Collecting, analyzing and gaining insight from large volumes of data is now\nthe norm in an ever increasing number of industries. Data analytics techniques,\nsuch as machine learning, are powerful tools used to analyze these large\nvolumes of data. Synthetic data sets are routinely relied upon to train and\ndevelop such data analytics methods for several reasons: to generate larger\ndata sets than are available, to generate diverse data sets, to preserve\nanonymity in data sets with sensitive information, etc. Processing,\ntransmitting and storing data is a key issue faced when handling large data\nsets. This paper presents an "On the fly" framework for generating big\nsynthetic data sets, suitable for these data analytics methods, that is both\ncomputationally efficient and applicable to a diverse set of problems. An\nexample application of the proposed framework is presented along with a\nmathematical analysis of its computational efficiency, demonstrating its\neffectiveness.\n