Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs

2023/10/11 by Abhinav Rao, Rao, Abhinav, Aditi Khandelwal +7 · 28 citations
Economics, Econometrics and Finance · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Law #Computation and Language (cs.CL) #FOS: Computer and information sciences #Law, Economics, and Judicial Systems #Legal Education and Practice Innovations

paper · pdf · doi:10.48550/arxiv.2310.07251

openalex publication_date 2023/10/11 · openalex created_date 2023/10/13 · openalex updated_date 2026/07/28

Abstract

In this position paper, we argue that instead of morally aligning LLMs to specific set of ethical principles, we should infuse generic ethical reasoning capabilities into them so that they can handle value pluralism at a global scale. When provided with an ethical policy, an LLM should be capable of making decisions that are ethically consistent to the policy. We develop a framework that integrates moral dilemmas with moral principles pertaining to different foramlisms of normative ethics, and at different levels of abstractions. Initial experiments with GPT-x models shows that while GPT-4 is a nearly perfect ethical reasoner, the models still have bias towards the moral values of Western and English speaking societies.

Cited by

• Morality in AI. A plea to embed morality in LLM architectures and frameworks
• Diverse Human Value Alignment for Large Language Models via Ethical Reasoning
• Implicit Values Embedded in How Humans and LLMs Complete Subjective Everyday Tasks
• Advancing Automated Ethical Profiling in SE: a Zero-Shot Evaluation of LLM Reasoning
• Pragmatic Inference for Moral Reasoning Acquisition: Generalization via Metapragmatic Links
• Exploring psychological tradeoffs: Developing and demonstrating an R Shiny app for Pareto optimization
• How do LLMs reflect human moral foundations? a study using the moral foundations framework
• Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
• Established Psychometric vs. Ecologically Valid Questionnaires: Rethinking Psychological Assessments in Large Language Models
• Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
• PROPS: Progressively Private Self-alignment of Large Language Models
• Dynamite: Real-Time Debriefing Slide Authoring through AI-Enhanced Multimodal Interaction
• Synthetic Voice Data for Automatic Speech Recognition in African Languages
• User Behavior Prediction as a Generic, Robust, Scalable, and Low-Cost Evaluation Strategy for Estimating Generalization in LLMs
• The Confidence Paradox: Can LLM Know When It's Wrong
• Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation
• Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models
• Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
• ProRefine: Inference-Time Prompt Refinement with Textual Feedback
• The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process
• Deontological Keyword Bias: The Impact of Modal Expressions on Normative Judgments of Language Models
• Are Language Models Consequentialist or Deontological Moral Reasoners?
• From Word to World: Evaluate and Mitigate Culture Bias in LLMs via Word Association Test
• OpenEthics: A Comprehensive Ethical Evaluation of Open-Source Generative Large Language Models
• Bridging AI and Carbon Capture: A Dataset for LLMs in Ionic Liquids and CBE Research
• The Fragility Of Moral Judgment In Large Language Models
• Moral Reasoning Across Languages: The Critical Role of Low-Resource Languages in LLMs
• Learning to Diagnose and Correct Errors: Towards Moral Sensitivity Acquisition in Large Language Models

Related

• The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas
• "Pull or Not to Pull?'': Investigating Moral Biases in Leading Large Language Models Across Ethical Dilemmas
• The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas
• Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistants
• Ethical Reasoning and Moral Value Alignment of LLMs Depend on the Language we Prompt them in