2016/07/04 by Manuel Alfonseca, Manuel Cebrián, Manuel Cebrian +6 · 2 voices · 58 citations
Computer Science · Decision Sciences · #Artificial intelligence #Computability #Computability, Logic, AI Algorithms #Computer science #Containment (computer programming) #Epistemology #Philosophy #Programming language #Scientific Computing and Data Management #State (computer science) #TRACE (psycholinguistics) #Theoretical computer science #Turing #Turing machine #cs.AI #cs.CY
paper · pdf · open access · doi:10.1613/jair.1.12202
published in Journal of Artificial Intelligence Research 70, 65-76 (AI Access Foundation) · 7 pages, 5 figures
arxiv created 2016/07/04 · openalex publication_date 2021/01/05 · arxiv updated 2021/01/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potential catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible.