2021/08/30 by Lijie Wang, Wang, Lijie, Hao Liu +13 · 3 citations
Computer Science · Medicine · #Adversarial Robustness in Machine Learning #Artificial Intelligence in Healthcare and Education #Artificial intelligence #Computer science #Computer security #Data mining #Data science #Deep learning #Deep neural networks #Explainable Artificial Intelligence (XAI) #Interpretability #Machine learning #Robustness (evolution) #Sentiment analysis #Trustworthiness #cs.AI #cs.CL
paper · pdf · doi:10.48550/arxiv.2108.13140
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2021/08/30 · arxiv created 2021/09/07 · arxiv updated 2021/09/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
While deep learning models have greatly improved the performance of most artificial intelligence tasks, they are often criticized to be untrustworthy due to the black-box problem. Consequently, many works have been proposed to study the trustworthiness of deep learning. However, as most open datasets are designed for evaluating the accuracy of model outputs, there is still a lack of appropriate datasets for evaluating the inner workings of neural networks. The lack of datasets obviously hinders the development of trustworthiness research. Therefore, in order to systematically evaluate the factors for building trustworthy systems, we propose a novel and well-annotated sentiment analysis dataset to evaluate robustness and interpretability. To evaluate these factors, our dataset contains diverse annotations about the challenging distribution of instances, manual adversarial instances and sentiment explanations. Several evaluation metrics are further proposed for interpretability and robustness. Based on the dataset and metrics, we conduct comprehensive comparisons for the trustworthiness of three typical models, and also study the relations between accuracy, robustness and interpretability. We release this trustworthiness evaluation dataset at \urlhttps://github/xyz and hope our work can facilitate the progress on building more trustworthy systems for real-world applications.