2019/01/01 by Roman V. Yampolskiy, Yampolskiy, Roman V. · 2 citations
Neuroscience · Psychology · Social Sciences · #Artificial Intelligence (cs.AI) #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Human-Automation Interaction and Safety #Psychology of Moral and Emotional Judgment
paper · pdf · doi:10.48550/arxiv.1901.01851
openalex publication_date 2019/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
AI Safety researchers attempting to align values of highly capable intelligent systems with those of humanity face a number of challenges including personal value extraction, multi-agent value merger and finally in-silico encoding. State-of-the-art research in value alignment shows difficulties in every stage in this process, but merger of incompatible preferences is a particularly difficult challenge to overcome. In this paper we assume that the value extraction problem will be solved and propose a possible way to implement an AI solution which optimally aligns with individual preferences of each user. We conclude by analyzing benefits and limitations of the proposed approach.