On Controllability of AI

2020/07/19 by Roman V. Yampolskiy, Yampolskiy, Roman V. · 5 citations
Computer Science · Mathematics · Neuroscience · Physics and Astronomy · Social Sciences · #Applications of artificial intelligence #Artificial Intelligence (cs.AI) #Artificial general intelligence #Artificial intelligence #Business #Civilization #Computer science #Computers and Society (cs.CY) #Control (management) #Controllability #Epistemology #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Humanity #Law #Mathematics #Neuroethics, Human Enhancement, Biomedical Innovations #Order (exchange) #Philosophy #Political science #Risk analysis (engineering) #Space Science and Extraterrestrial Life #cs.AI #cs.CY

paper · pdf · doi:10.48550/arxiv.2008.04071

published in arXiv (Cornell University) (Cornell University)

arxiv created 2020/07/19 · openalex publication_date 2020/07/19 · arxiv updated 2020/08/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Invention of artificial general intelligence is predicted to cause a shift in the trajectory of human civilization. In order to reap the benefits and avoid pitfalls of such powerful technology it is important to be able to control it. However, possibility of controlling artificial general intelligence and its more advanced version, superintelligence, has not been formally established. In this paper, we present arguments as well as supporting evidence from multiple domains indicating that advanced AI can't be fully controlled. Consequences of uncontrollability of AI are discussed with respect to future of humanity and research on AI, and AI safety and security.

Citations

• Cybernetics or Control and Communication in the Animal and the Machine
• Information-theoretic approach to the study of control systems
• Impossibility of distributed consensus with one faulty process
• Universal Intelligence: A Definition of Machine Intelligence
• Artificial intelligence: a modern approach
• On Computable Numbers, with an Application to the Entscheidungsproblem
• Superintelligence cannot be contained: Lessons from Computability Theory
• The impossibility of "fairness": a generalized impossibility result for decisions
• Constraints on physical reality arising from a formalization of knowledge
• Interstellar communication. IX. Message decontamination is impossible
• Building Safer AGI by introducing Artificial Stupidity
• Stovepiping and Malicious Software: A Critical Review of AGI Containment
• Scalable agent alignment via reward modeling: a research direction
• Learnability can be undecidable
• Personal Universes: A Solution to the Multi-Agent Value Alignment Problem
• Risks from Learned Optimization in Advanced Machine Learning Systems
• Language Models are Few-Shot Learners
• AI Research Considerations for Human Existential Safety (ARCHES)

Cited by

• The Decision Path to Control AI Risks Completely: Fundamental Control Mechanisms for AI Governance

Related

• The three main doctrines on the future of AI
• AI Must not be Fully Autonomous
• Human-controllable AI: Meaningful Human Control
• The Decision Path to Control AI Risks Completely: Fundamental Control Mechanisms for AI Governance
• Reframing AI Loss of Control: What Control Is, How to Have It, How to Lose It