2025/12/24 by Khondoker Mirazul Mumenin, Robert Underwood, Mumenin, Khondoker Mirazul +11
Computer Science · Decision Sciences · #Advanced Data Compression Techniques #Advanced Data Storage Technologies #Data Quality and Management #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Performance (cs.PF) #and Cluster Computing (cs.DC)
paper · doi:10.48550/arxiv.2512.21433
openalex publication_date 2025/12/24 · openalex created_date 2025/12/30 · openalex updated_date 2026/07/28
Error-bounded lossy compression techniques have become vital for scientific data management and analytics, given the ever-increasing volume of data generated by modern scientific simulations and instruments. Nevertheless, assessing data quality post-compression remains computationally expensive due to the intensive nature of metric calculations. In this work, we present a general-purpose deep-surrogate framework for lossy compression quality prediction (DeepCQ), with the following key contributions: 1) We develop a surrogate model for compression quality prediction that is generalizable to different error-bounded lossy compressors, quality metrics, and input datasets; 2) We adopt a novel two-stage design that decouples the computationally expensive feature-extraction stage from the light-weight metrics prediction, enabling efficient training and modular inference; 3) We optimize the model performance on time-evolving data using a mixture-of-experts design. Such a design enhances the robustness when predicting across simulation timesteps, especially when the training and test data exhibit significant variation. We validate the effectiveness of DeepCQ on four real-world scientific applications. Our results highlight the framework's exceptional predictive accuracy, with prediction errors generally under 10% across most settings, significantly outperforming existing methods. Our framework empowers scientific users to make informed decisions about data compression based on their preferred data quality, thereby significantly reducing I/O and computational overhead in scientific data analysis.