2024/04/30 by Swaminathan, Nandhini, Danks, David
#68T05 #91A #Artificial Intelligence (cs.AI) #Computer Science and Game Theory (cs.GT) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #I.2.0 #Machine Learning (cs.LG) #Multiagent Systems (cs.MA)
paper · doi:10.48550/arxiv.2404.19256
One strategy in response to pluralistic values in a user population is to personalize an AI system: if the AI can adapt to the specific values of each individual, then we can potentially avoid many of the challenges of pluralism. Unfortunately, this approach creates a significant ethical issue: if there is an external measure of success for the human-AI team, then the adaptive AI system may develop strategies (sometimes deceptive) to compensate for its human teammate. This phenomenon can be viewed as a form of social compensation, where the AI makes decisions based not on predefined goals but on its human partner's deficiencies in relation to the team's performance objectives. We provide a practical ethical analysis of the conditions in which such compensation may nonetheless be justifiable.