2021/09/01 by Hossein Amirkhani, Amirkhani, Hossein, Mohammad Taher Pilehvar +1 · 1 citation
Computer Science · #Bayesian Modeling and Causal Inference #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
paper · pdf · doi:10.48550/arxiv.2109.00521
openalex publication_date 2021/09/01 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Existing techniques for mitigating dataset bias often leverage a biased model\nto identify biased instances. The role of these biased instances is then\nreduced during the training of the main model to enhance its robustness to\nout-of-distribution data. A common core assumption of these techniques is that\nthe main model handles biased instances similarly to the biased model, in that\nit will resort to biases whenever available. In this paper, we show that this\nassumption does not hold in general. We carry out a critical investigation on\ntwo well-known datasets in the domain, MNLI and FEVER, along with two biased\ninstance detection methods, partial-input and limited-capacity models. Our\nexperiments show that in around a third to a half of instances, the biased\nmodel is unable to predict the main model's behavior, highlighted by the\nsignificantly different parts of the input on which they base their decisions.\nBased on a manual validation, we also show that this estimate is highly in line\nwith human interpretation. Our findings suggest that down-weighting of\ninstances detected by bias detection methods, which is a widely-practiced\nprocedure, is an unnecessary waste of training data. We release our code to\nfacilitate reproducibility and future research.\n