vix.ing · top · new · best · stats · spec

Online Detection of Sparse Changes in High-Dimensional Data Streams\n Using Tailored Projections

2019/08/06 by Martin Tveten, Tveten, Martin, Ingrid K. Glad +1 · 1 citation
Engineering · Decision Sciences · Chemistry · #Fault Detection and Control Systems #Advanced Statistical Process Monitoring #Spectroscopy and Chemometric Analyses

paper · pdf · doi:10.48550/arxiv.1908.02029

Abstract

When applying principal component analysis (PCA) for dimension reduction, the\nmost varying projections are usually used in order to retain most of the\ninformation. For the purpose of anomaly and change detection, however, the\nleast varying projections are often the most important ones. In this article,\nwe present a novel method that automatically tailors the choice of projections\nto monitor for sparse changes in the mean and/or covariance matrix of\nhigh-dimensional data. A subset of the least varying projections is almost\nalways selected based on a criteria of the projection's sensitivity to changes.\n Our focus is on online/sequential change detection, where the aim is to\ndetect changes as quickly as possible, while controlling false alarms at a\nspecified level. A combination of tailored PCA and a generalized log-likelihood\nmonitoring procedure displays high efficiency in detecting even very sparse\nchanges in the mean, variance and correlation. We demonstrate on real data that\ntailored PCA monitoring is efficient for sparse change detection also when the\ndata streams are highly auto-correlated and non-normal. Notably, error control\nis achieved without a large validation set, which is needed in most existing\nmethods.\n

Cited by

Related