Improving the State of the Art for Training Human-AI Teams: Technical Report #5 -- Individual Differences and Team Qualities to Measure in a Human-AI Teaming Testbed

2025/06/05 by Asiala, Lillian, McCarthy, James E.
#FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)

paper · doi:10.48550/arxiv.2507.18878

Abstract

Sonalysts, Inc. (Sonalysts) is working on an initiative to expand our expertise in teaming to include Human-Artificial Intelligence (AI) teams. The first step of this process is to develop a Synthetic Task Environment (STE) to support our original research. Prior knowledge elicitation efforts within the Human-AI teaming research stakeholder community revealed a desire to support data collection using pre- and post-performance surveys. In this technical report, we review a number of constructs that capture meaningful individual differences and teaming qualities. Additionally, we explore methods of measuring those constructs within the STE.

Related

• Improving the State of the Art for Training Human-AI Teams: Technical Report #3 -- Analysis of Testbed Alternatives
• Improving the State of the Art for Training Human-AI Teams: Technical Report #2 -- Results of Researcher Knowledge Elicitation Survey
• Improving the State of the Art for Training Human-AI Teams: Technical Report #1 -- Results of Subject-Matter Expert Knowledge Elicitation Survey
• CREW: Facilitating Human-AI Teaming Research
• Unraveling Human-AI Teaming: A Review and Outlook