Provably safe systems: the only path to controllable AGI

2023/09/05 by Max Tegmark, Steve Omohundro, Tegmark, Max +1 · 2 voices · 11 citations
#cs.CY #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2309.01933

Abstract

We describe a path to humanity safely thriving with powerful Artificial General Intelligences (AGIs) by building them to provably satisfy human-specified requirements. We argue that this will soon be technically feasible using advanced AI for formal verification and mechanistic interpretability. We further argue that it is the only path which guarantees safe controlled AGI. We end with a list of challenge problems whose solution would contribute to this positive outcome and invite readers to join in this work.

Cited by

• International Agreements to Limit Frontier AI: Objectives and Exit
• Distributional AGI Safety
• STAMP/STPA informed characterization of Factors Leading to Loss of Control in AI Systems
• Verifying Memoryless Sequential Decision-making of Large Language Models
• Logical Consistency Between Disagreeing Experts and Its Role in AI Safety
• ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attack
• Limitations on Safe, Trusted, Artificial General Intelligence
• Bounded PCTL Model Checking of Large Language Model Outputs
• No-Knowledge Alarms for Misaligned LLMs-as-Judges
• AI Testing Should Account for Sophisticated Strategic Behaviour
• Lessons from complex systems science for AI governance

Discussions

• Provably safe systems: the only path to controllable AGI [hn, 45 points, 64 comments]
• Provably safe systems: the only path to controllable AGI [hn, 2 points, 1 comments]

Related

• Limitations on Safe, Trusted, Artificial General Intelligence
• Towards AI-45^∘ Law: A Roadmap to Trustworthy AGI
• Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
• Safe AI -- How is this Possible?
• Building Safer AGI by introducing Artificial Stupidity