Unpredictability of AI: On the Impossibility of Accurately Predicting All Actions of a Smarter Agent

2020/03/01 by Roman V. Yampolskiy · 44 citations
Computer Science · Mathematics · #Artificial intelligence #Bayesian Modeling and Causal Inference #Business #Computability, Logic, AI Algorithms #Computer science #Computer security #Data science #Field (mathematics) #Impossibility #Mathematics #Political science #Process (computing) #Reinforcement Learning in Robotics #Risk analysis (engineering)

paper · doi:10.1142/s2705078520500034

published in Journal of Artificial Intelligence and Consciousness 07(01), 109-118 (World Scientific)

openalex publication_date 2020/03/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/05/21

Abstract

The young field of AI Safety is still in the process of identifying its challenges and limitations. In this paper, we formally describe one such impossibility result, namely Unpredictability of AI. We prove that it is impossible to precisely and consistently predict what specific actions a smarter-than-human intelligent system will take to achieve its objectives, even if we know the terminal goals of the system. In conclusion, the impact of Unpredictability on AI Safety is discussed.

Citations

• Impossibility of distributed consensus with one faulty process
• Classes of recursively enumerable sets and their decision problems
• Deep Blue

Cited by

• Ethics of artificial intelligence [wikipedia]