Trusting Automation: Designing for Responsivity and Resilience
2021/04/27 by Erin K. Chiou, John D. Lee · 282 citations
Decision Sciences · Engineering · Psychology · #Automation #Business #Cognitive Functions and Memory #Computer science #Engineering #Framing (construction) #Human-Automation Interaction and Safety #ISA100.11a #Knowledge management #Personal Information Management and User Behavior #Process automation system #Process management #Risk analysis (engineering) #Situation awareness
paper · doi:10.1177/00187208211009995
published in Human Factors The Journal of the Human Factors and Ergonomics Society 65(1), 137-165 (SAGE Publishing)
openalex publication_date 2021/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04
Abstract
OBJECTIVE: This paper reviews recent articles related to human trust in automation to guide research and design for increasingly capable automation in complex work environments. BACKGROUND: Two recent trends-the development of increasingly capable automation and the flattening of organizational hierarchies-suggest a reframing of trust in automation is needed. METHOD: Many publications related to human trust and human-automation interaction were integrated in this narrative literature review. RESULTS: Much research has focused on calibrating human trust to promote appropriate reliance on automation. This approach neglects relational aspects of increasingly capable automation and system-level outcomes, such as cooperation and resilience. To address these limitations, we adopt a relational framing of trust based on the decision situation, semiotics, interaction sequence, and strategy. This relational framework stresses that the goal is not to maximize trust, or to even calibrate trust, but to support a process of trusting through automation responsivity. CONCLUSION: This framing clarifies why future work on trust in automation should consider not just individual characteristics and how automation influences people, but also how people can influence automation and how interdependent interactions affect trusting automation. In these new technological and organizational contexts that shift human operators to co-operators of automation, automation responsivity and the ability to resolve conflicting goals may be more relevant than reliability and reliance for advancing system design. APPLICATION: A conceptual model comprising four concepts-situation, semiotics, strategy, and sequence-can guide future trust research and design for automation responsivity and more resilient human-automation systems.
Citations
Cited by
- Disuse of advanced driver assistance systems (ADAS)
- The role of critical thinking on undergraduates' reliance behaviours on generative AI in problem‐solving
- From Binary to Bilingual: How the National Weather Service is Using Artificial Intelligence to Develop a Comprehensive Translation Program
- Twenty-four years of empirical research on trust in AI: a bibliometric review of trends, overlooked issues, and future directions
- Artificial intelligence for collective intelligence: a national-scale research strategy
- Learning to Complement with Multiple Humans
- Trust and trustworthy artificial intelligence: A research agenda for AI in the environmental sciences
- Grand Challenges around Designing Computers’ Control Over Our Bodies
- Calibrating workers’ trust in intelligent automated systems
- The evolution of professional identity: How academic librarians perceive and adapt to the rise of generative AI in their daily work
- Effects of automation trust in drivers' visual distraction during automation. [europepmc]
- Measurement of Trust in Automation: A Narrative Review and Reference Guide. [europepmc]
- Human Factors Considerations and Metrics in Shared Space Human-Robot Collaboration: A Systematic Review. [europepmc]
- Extended Interviews with Stroke Patients Over a Long-Term Rehabilitation Using Human-Robot or Human-Computer Interactions. [europepmc]
- Motivation to interaction media: The impact of automation trust and self-determination theory on intention to use the new interaction technology in autonomous vehicles. [europepmc]
- From Teams to Teamness: Future Directions in the Science of Team Cognition. [europepmc]
- The Relationships of Human-Cobot Interaction Fluency with Job Performance and Job Satisfaction among Cobot Operators-The Moderating Role of Workload. [europepmc]
- How Terminology Affects Users' Responses to System Failures. [europepmc]
- Measuring trust: a text analysis approach to compare, contrast, and select trust questionnaires. [europepmc]
- A review of mathematical models of human trust in automation. [europepmc]
- Neural Correlates of Trust in Automation: Considerations and Generalizability Between Technology Domains. [europepmc]
- Using fNIRS to Identify Transparency- and Reliability-Sensitive Markers of Trust Across Multiple Timescales in Collaborative Human-Human-Agent Triads. [europepmc]
- Assessing Patient Trust in Automation in Health Care Systems: Within-Subjects Experimental Study. [europepmc]
- Overtrust in AI Recommendations About Whether or Not to Kill: Evidence from Two Human-Robot Interaction Studies. [europepmc]
- How Explainable Artificial Intelligence Can Increase or Decrease Clinicians' Trust in AI Applications in Health Care: Systematic Review. [europepmc]
- Physical embodiment and anthropomorphism of AI tutors and their role in student enjoyment and performance. [europepmc]
- Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists. [europepmc]
- Influence of AI behavior on human moral decisions, agency, and responsibility. [europepmc]
- Can humanoid robots be used as a cognitive offloading tool? [europepmc]
- Failing to Grasp our Failure to Grasp Automation Failure. [europepmc]
- Deepening ideas vs. exploring new ones: AI strategy effects in human-AI creative collaboration. [europepmc]
- How Multi-Tasking Ability Impacts Performance, Workload, Situation Awareness, Stress and Trust with Simulated Imperfect Automation. [europepmc]
- Toward a science of human-AI teaming for decision making: A complementarity framework. [europepmc]
- From Initial to Situational Automation Trust: The Interplay of Personality, Interpersonal Trust, and Trust Calibration in Young Males [europepmc]
- Using Generative AI in Learning and Students’ Innovative Behavior: A Dual-Path Examination Based on the UTAUT Model [europepmc]
Related