Trusting Automation: Designing for Responsivity and Resilience
2021/04/27 by Erin K. Chiou, John D. Lee · 32 citations
Psychology · Decision Sciences · #Human-Automation Interaction and Safety #Personal Information Management and User Behavior #Cognitive Functions and Memory
paper · doi:10.1177/00187208211009995
Abstract
OBJECTIVE: This paper reviews recent articles related to human trust in automation to guide research and design for increasingly capable automation in complex work environments. BACKGROUND: Two recent trends-the development of increasingly capable automation and the flattening of organizational hierarchies-suggest a reframing of trust in automation is needed. METHOD: Many publications related to human trust and human-automation interaction were integrated in this narrative literature review. RESULTS: Much research has focused on calibrating human trust to promote appropriate reliance on automation. This approach neglects relational aspects of increasingly capable automation and system-level outcomes, such as cooperation and resilience. To address these limitations, we adopt a relational framing of trust based on the decision situation, semiotics, interaction sequence, and strategy. This relational framework stresses that the goal is not to maximize trust, or to even calibrate trust, but to support a process of trusting through automation responsivity. CONCLUSION: This framing clarifies why future work on trust in automation should consider not just individual characteristics and how automation influences people, but also how people can influence automation and how interdependent interactions affect trusting automation. In these new technological and organizational contexts that shift human operators to co-operators of automation, automation responsivity and the ability to resolve conflicting goals may be more relevant than reliability and reliance for advancing system design. APPLICATION: A conceptual model comprising four concepts-situation, semiotics, strategy, and sequence-can guide future trust research and design for automation responsivity and more resilient human-automation systems.
Citations
Cited by
- Disuse of advanced driver assistance systems (ADAS)
- The role of critical thinking on undergraduates' reliance behaviours on generative <scp>AI</scp> in problem‐solving
- From Binary to Bilingual: How the National Weather Service is Using Artificial Intelligence to Develop a Comprehensive Translation Program
- Twenty-four years of empirical research on trust in AI: a bibliometric review of trends, overlooked issues, and future directions
- Artificial intelligence for collective intelligence: a national-scale research strategy
- Learning to Complement with Multiple Humans
- Trust and trustworthy artificial intelligence: A research agenda for AI in the environmental sciences
- Effects of automation trust in drivers' visual distraction during automation. [europepmc]
- Measurement of Trust in Automation: A Narrative Review and Reference Guide. [europepmc]
- Human Factors Considerations and Metrics in Shared Space Human-Robot Collaboration: A Systematic Review. [europepmc]
- Extended Interviews with Stroke Patients Over a Long-Term Rehabilitation Using Human-Robot or Human-Computer Interactions. [europepmc]
- Motivation to interaction media: The impact of automation trust and self-determination theory on intention to use the new interaction technology in autonomous vehicles. [europepmc]
- From Teams to Teamness: Future Directions in the Science of Team Cognition. [europepmc]
- The Relationships of Human-Cobot Interaction Fluency with Job Performance and Job Satisfaction among Cobot Operators-The Moderating Role of Workload. [europepmc]
- How Terminology Affects Users' Responses to System Failures. [europepmc]
- Measuring trust: a text analysis approach to compare, contrast, and select trust questionnaires. [europepmc]
- A review of mathematical models of human trust in automation. [europepmc]
- Neural Correlates of Trust in Automation: Considerations and Generalizability Between Technology Domains. [europepmc]
- Using fNIRS to Identify Transparency- and Reliability-Sensitive Markers of Trust Across Multiple Timescales in Collaborative Human-Human-Agent Triads. [europepmc]
- Assessing Patient Trust in Automation in Health Care Systems: Within-Subjects Experimental Study. [europepmc]
- Overtrust in AI Recommendations About Whether or Not to Kill: Evidence from Two Human-Robot Interaction Studies. [europepmc]
- How Explainable Artificial Intelligence Can Increase or Decrease Clinicians' Trust in AI Applications in Health Care: Systematic Review. [europepmc]
- Physical embodiment and anthropomorphism of AI tutors and their role in student enjoyment and performance. [europepmc]
- Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists. [europepmc]
- Influence of AI behavior on human moral decisions, agency, and responsibility. [europepmc]
- Can humanoid robots be used as a cognitive offloading tool? [europepmc]
- Failing to Grasp our Failure to Grasp Automation Failure. [europepmc]
- Deepening ideas vs. exploring new ones: AI strategy effects in human-AI creative collaboration. [europepmc]
- How Multi-Tasking Ability Impacts Performance, Workload, Situation Awareness, Stress and Trust with Simulated Imperfect Automation. [europepmc]
- Toward a science of human-AI teaming for decision making: A complementarity framework. [europepmc]
- From Initial to Situational Automation Trust: The Interplay of Personality, Interpersonal Trust, and Trust Calibration in Young Males [europepmc]
- Using Generative AI in Learning and Students’ Innovative Behavior: A Dual-Path Examination Based on the UTAUT Model [europepmc]
Related