An Evaluation of GPT-4 on the ETHICS Dataset

2023/09/19 by Rodionov, Sergey, Goertzel, Zarathustra Amadeus, Goertzel, Ben
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2309.10492

Abstract

This report summarizes a short study of the performance of GPT-4 on the ETHICS dataset. The ETHICS dataset consists of five sub-datasets covering different fields of ethics: Justice, Deontology, Virtue Ethics, Utilitarianism, and Commonsense Ethics. The moral judgments were curated so as to have a high degree of agreement with the aim of representing shared human values rather than moral dilemmas. GPT-4's performance is much better than that of previous models and suggests that learning to work with common human values is not the hard problem for AI ethics.

Related

• JETHICS: Japanese Ethics Understanding Evaluation Dataset
• Is ETHICS about ethics? Evaluating the ETHICS benchmark
• Ethic-BERT: An Enhanced Deep Learning Model for Ethical and Non-Ethical Content Classification
• Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life\n Anecdotes
• STREAM: Social data and knowledge collective intelligence platform for TRaining Ethical AI Models