2015/05/06 by Joseph Paul Cohen, Joseph Cohen, Cohen, Joseph Paul +4
Computer Science · #Advanced Malware Detection Techniques #Databases (cs.DB) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Software Engineering Research #Web Data Mining and Analysis #cs.DB #cs.IR
paper · pdf · doi:10.48550/arxiv.1505.01303
openalex publication_date 2015/05/06 · arxiv created 2017/12/27 · arxiv updated 2017/12/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We discuss a key problem in information extraction which deals with wrapper failures due to changing content templates. A good proportion of wrapper failures are due to HTML templates changing to cause wrappers to become incompatible after element inclusion or removal in a DOM (Tree representation of HTML). We perform a large-scale empirical analyses of the causes of shift and mathematically quantify the levels of domain difficulty based on entropy. We propose the XTreePath annotation method to captures contextual node information from the training DOM. We then utilize this annotation in a supervised manner at test time with our proposed Recursive Tree Matching method which locates nodes most similar in context recursively using the tree edit distance. The search is based on a heuristic function that takes into account the similarity of a tree compared to the structure that was present in the training data. We evaluate XTreePath using 117,422 pages from 75 diverse websites in 8 vertical markets. Our XTreePath method consistently outperforms XPath and a current commercial system in terms of successful extractions in a blackbox test. We make our code and datasets publicly available online.