vix.ing · top · new · best · stats · spec

Cleaning data with Swipe

2024/03/28 by Toon Boeckling, Boeckling, Toon, Antoon Bronselaer +1
Computer Science · #Data Mining Algorithms and Applications #Databases (cs.DB) #Digital and Cyber Forensics #FOS: Computer and information sciences #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2403.19378

openalex publication_date 2024/03/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The repair problem for functional dependencies is the problem where an input database needs to be modified such that all functional dependencies are satisfied and the difference with the original database is minimal. The output database is then called an optimal repair. If the allowed modifications are value updates, finding an optimal repair is NP-hard. A well-known approach to find approximations of optimal repairs builds a Chase tree in which each internal node resolves violations of one functional dependency and leaf nodes represent repairs. A key property of this approach is that controlling the branching factor of the Chase tree allows to control the trade-off between repair quality and computational efficiency. In this paper, we explore an extreme variant of this idea in which the Chase tree has only one path. To construct this path, we first create a partition of attributes such that classes can be repaired sequentially. We repair each class only once and do so by fixing the order in which dependencies are repaired. This principle is called priority repairing and we provide a simple heuristic to determine priority. The techniques for attribute partitioning and priority repair are combined in the Swipe algorithm. An empirical study on four real-life data sets shows that Swipe is one to three orders of magnitude faster than multi-sequence Chase-based approaches, whereas the quality of repairs is comparable or better. Moreover, a scalability analysis of the Swipe algorithm shows that Swipe scales well in terms of an increasing number of tuples.

Related