2024/06/15 by Akash V. Maharaj, Kun Qian, Maharaj, Akash V. +18 · 1 citation
Engineering · #Digital Transformation in Industry
paper · pdf · doi:10.48550/arxiv.2407.12003
The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.