2018/07/09 by Erich Schubert, Michael Gertz · 2 citations
Economics, Econometrics and Finance · Computer Science · #Complex Systems and Time Series Analysis #Data Visualization and Analytics #Time Series Analysis and Forecasting
paper · doi:10.1145/3221269.3223036
openalex publication_date 2018/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
With the advent of big data, we see an increasing interest in computing correlations in huge data sets with both many instances and many variables. Essential descriptive statistics such as the variance, standard deviation, covariance, and correlation can suffer from a numerical instability known as "catastrophic cancellation" that can lead to problems when naively computing these statistics with a popular textbook equation. While this instability has been discussed in the literature already 50 years ago, we found that even today, some high-profile tools still employ the instable version.