Measuring Goal-Directedness

2024/12/06 by Matt MacDermott, MacDermott, Matt, James Fox +5 · 1 voice · 4 citations
Computer Science · Decision Sciences · #Evaluation and Performance Assessment #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2412.04758

Abstract

We define maximum entropy goal-directedness (MEG), a formal measure of goal-directedness in causal models and Markov decision processes, and give algorithms for computing it. Measuring goal-directedness is important, as it is a critical element of many concerns about harm from AI. It is also of philosophical interest, as goal-directedness is a key aspect of agency. MEG is based on an adaptation of the maximum causal entropy framework used in inverse reinforcement learning. It can measure goal-directedness with respect to a known utility function, a hypothesis class of utility functions, or a set of random variables. We prove that MEG satisfies several desiderata and demonstrate our algorithms with small-scale experiments.

Cited by

• A Principle of Targeted Intervention for Multi-Agent Reinforcement Learning
• Goal-Directedness is in the Eye of the Beholder
• The Singapore Consensus on Global AI Safety Research Priorities
• Evaluating the Goal-Directedness of Large Language Models

Discussions

• Come and chat to us about it at the poster session. 4.30-7.30pm. East Exhibit Hall A-C #3000. 🔗 Link to the full paper: arxiv.org/pdf/2412.04758 [bsky, 0 points, 1 comments]

Related

• Towards Measuring Goal-Directedness in AI Systems
• Goal-Directedness is in the Eye of the Beholder
• Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learning
• Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
• Maximum Causal Entropy Inverse Constrained Reinforcement Learning