2025/07/08 by Julian Fortune, Fortune, Julian, Julie A. Adams +3
Computer Science · Engineering · Psychology · #Aerospace and Aviation Technology #FOS: Computer and information sciences #Human-Automation Interaction and Safety #Machine Learning (cs.LG) #Robotics (cs.RO) #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.2507.05985
openalex publication_date 2025/07/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Demanding task environments (e.g., supervising a remotely piloted aircraft) require performing tasks quickly and accurately; however, periods of low and high operator workload can decrease task performance. Intelligent modulation of the system's demands and interaction modality in response to changes in operator workload state may increase performance by avoiding undesirable workload states. This system requires real-time estimation of each workload component (i.e., cognitive, physical, visual, speech, and auditory) to adapt the correct modality. Existing workload systems estimate multiple workload components post-hoc, but few estimate speech workload, or function in real-time. An algorithm to estimate speech workload and mitigate undesirable workload states in real-time is presented. An analysis of the algorithm's accuracy is presented, along with the results demonstrating the algorithm's generalizability across individuals and human-machine teaming paradigms. Real-time speech workload estimation is a crucial element towards developing adaptive human-machine systems.