Fast estimation of the ICL criterion for change-point detection problems with applications to Next-Generation Sequencing data

2012/11/14 by Alice Cleynen, Cleynen, Alice, The Minh Luong +5 · 1 citation
Mathematics · #Applications (stat.AP) #Computation (stat.CO) #FOS: Computer and information sciences #stat.AP #stat.CO

paper · pdf · doi:10.48550/arxiv.1211.3210

15 pages, 8 figures

arxiv created 2013/07/01 · arxiv updated 2013/07/02

Abstract

In this paper, we consider the Integrated Completed Likelihood (ICL) as a useful criterion for estimating the number of changes in the underlying distribution of data in problems where detecting the precise location of these changes is the main goal. The exact computation of the ICL requires O(Kn2) operations (with K the number of segments and n the number of data-points) which is prohibitive in many practical situations with large sequences of data. We describe a framework to estimate the ICL with O(Kn) complexity. Our approach is general in the sense that it can accommodate any given model distribution. We checked the run-time and validity of our approach on simulated data and demonstrate its good performance when analyzing real Next-Generation Sequencing (NGS) data using a negative binomial model.

Cited by

• Extending finite mixture models with skew-normal distributions and hidden Markov models for time series

Related

• Optimal detection of changepoints with a linear computational cost
• A novel changepoint detection algorithm
• Fast estimation of posterior probabilities in change-point models through a constrained hidden Markov model
• Minimizing Change-Point Estimation Error
• Sequential change-point detection: Computation versus statistical performance