Safety Features for a Centralised AGI Project

2025/06/17 by Hastings-Woodhouse, Sarah
#Computers and Society (cs.CY) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2507.21082

Abstract

Recent AI progress has outpaced expectations, with some experts now predicting AI that matches or exceeds human capabilities in all cognitive areas (AGI) could emerge this decade, potentially posing grave national and global security threats. AI development is currently occurring primarily in the private sector with minimal oversight. This report analyzes a scenario where the US government centralizes AGI development under its direct control, and identifies four high-level priorities and seven safety features to reduce risks.

Citations

• Technical Options for Flexible Hardware-Enabled Guarantees
• AI Behind Closed Doors: a Primer on The Governance of Internal Deployment
• Intolerable Risk Threshold Recommendations for Artificial Intelligence
• Why do Experts Disagree on Existential Risk and P(doom)? A Survey of AI Experts
• Alignment faking in large language models
• RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts
• Verification methods for international AI agreements
• Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs
• AI Sandbagging: Language Models can Strategically Underperform on Evaluations
• Training Compute Thresholds: Features and Functions in AI Regulation
• Affirmative safety: An approach to risk management for high-risk AI
• EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
• AI Supported Degradation of the Self Concept: A Theoretical Framework Grounded in Established Cognitive and Computational Mechanisms
• Provably safe systems: the only path to controllable AGI
• AI Deception: A Survey of Examples, Risks, and Potential Solutions
• Defining and Characterizing Reward Hacking
• Emergent Abilities of Large Language Models
• The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models
• Towards A Rigorous Science of Interpretable Machine Learning

Related

• AGI Safety Literature Review
• Towards best practices in AGI safety and governance: A survey of expert opinion
• Multinational AGI Consortium (MAGIC): A Proposal for International Coordination on AI
• International AI Safety Report
• Review of state-of-the-arts in artificial intelligence with application to AI safety problem