2021/09/09 by Carel van Niekerk, van Niekerk, Carel, Andrey Malinin +13 · 1 citation
Computer Science · #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Speech and dialogue systems #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2109.04349
openalex publication_date 2021/09/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The ability to identify and resolve uncertainty is crucial for the robustness\nof a dialogue system. Indeed, this has been confirmed empirically on systems\nthat utilise Bayesian approaches to dialogue belief tracking. However, such\nsystems consider only confidence estimates and have difficulty scaling to more\ncomplex settings. Neural dialogue systems, on the other hand, rarely take\nuncertainties into account. They are therefore overconfident in their decisions\nand less robust. Moreover, the performance of the tracking task is often\nevaluated in isolation, without consideration of its effect on the downstream\npolicy optimisation. We propose the use of different uncertainty measures in\nneural belief tracking. The effects of these measures on the downstream task of\npolicy optimisation are evaluated by adding selected measures of uncertainty to\nthe feature space of the policy and training policies through interaction with\na user simulator. Both human and simulated user results show that incorporating\nthese measures leads to improvements both of the performance and of the\nrobustness of the downstream dialogue policy. This highlights the importance of\ndeveloping neural dialogue belief trackers that take uncertainty into account.\n