2015/03/02 by Alexey Miroshnikov, Miroshnikov, Alexey, E. V. Savel'ev +3
Computer Science · Mathematics · #Applications (stat.AP) #Bayesian Methods and Mixture Models #Computation (stat.CO) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Statistical Methods and Bayesian Inference #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.1503.00635
openalex publication_date 2015/03/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent developments in data science and big data research have produced an\nabundance of large data sets that are too big to be analyzed in their entirety,\ndue to limits on either computer memory or storage capacity. Here, we introduce\nour R package 'BayesSummaryStatLM' for Bayesian linear regression models with\nMarkov chain Monte Carlo implementation that overcomes these limitations. Our\nBayesian models use only summary statistics of data as input; these summary\nstatistics can be calculated from subsets of big data and combined over\nsubsets. Thus, complete data sets do not need to be read into memory in full,\nwhich removes any physical memory limitations of a user. Our package\nincorporates the R package 'ff' and its functions for reading in big data sets\nin chunks while simultaneously calculating summary statistics. We describe our\nBayesian linear regression models, including several choices of prior\ndistributions for unknown model parameters, and illustrate capabilities and\nfeatures of our R package using both simulated and real data sets.\n