2020/12/15 by Udayan Khurana, Khurana, Udayan, Sainyam Galhotra +1 · 1 citation
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Data Management and Algorithms #Data Quality and Management #Databases (cs.DB) #FOS: Computer and information sciences #Time Series Analysis and Forecasting #cs.AI #cs.DB
paper · pdf · doi:10.48550/arxiv.2012.08594
arxiv created 2020/12/15 · openalex publication_date 2020/12/15 · arxiv updated 2020/12/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Detecting semantic concept of columns in tabular data is of particular interest to many applications ranging from data integration, cleaning, search to feature engineering and model building in machine learning. Recently, several works have proposed supervised learning-based or heuristic pattern-based approaches to semantic type annotation. Both have shortcomings that prevent them from generalizing over a large number of concepts or examples. Many neural network based methods also present scalability issues. Additionally, none of the known methods works well for numerical data. We propose C2, a column to concept mapper that is based on a maximum likelihood estimation approach through ensembles. It is able to effectively utilize vast amounts of, albeit somewhat noisy, openly available table corpora in addition to two popular knowledge graphs to perform effective and efficient concept prediction for structured data. We demonstrate the effectiveness of C2 over available techniques on 9 datasets, the most comprehensive comparison on this topic so far.