2018/09/12 by Jon Kleinberg, Sendhil Mullainathan, Kleinberg, Jon +1 · 1 citation
Economics, Econometrics and Finance · Social Sciences · #Computers and Society (cs.CY) #Corruption and Economic Development #Data Structures and Algorithms (cs.DS) #Economic Policies and Impacts #Experimental Behavioral Economics Studies #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Social and Information Networks (cs.SI)
paper · pdf · doi:10.48550/arxiv.1809.04578
openalex publication_date 2018/09/12 · openalex created_date 2022/08/03 · openalex updated_date 2026/07/28
Algorithms are increasingly used to aid, or in some cases supplant, human\ndecision-making, particularly for decisions that hinge on predictions. As a\nresult, two additional features in addition to prediction quality have\ngenerated interest: (i) to facilitate human interaction and understanding with\nthese algorithms, we desire prediction functions that are in some fashion\nsimple or interpretable; and (ii) because they influence consequential\ndecisions, we also want them to produce equitable allocations. We develop a\nformal model to explore the relationship between the demands of simplicity and\nequity. Although the two concepts appear to be motivated by qualitatively\ndistinct goals, we show a fundamental inconsistency between them. Specifically,\nwe formalize a general framework for producing simple prediction functions, and\nin this framework we establish two basic results. First, every simple\nprediction function is strictly improvable: there exists a more complex\nprediction function that is both strictly more efficient and also strictly more\nequitable. Put another way, using a simple prediction function both reduces\nutility for disadvantaged groups and reduces overall welfare relative to other\noptions. Second, we show that simple prediction functions necessarily create\nincentives to use information about individuals' membership in a disadvantaged\ngroup --- incentives that weren't present before simplification, and that work\nagainst these individuals. Thus, simplicity transforms disadvantage into bias\nagainst the disadvantaged group. Our results are not only about algorithms but\nabout any process that produces simple models, and as such they connect to the\npsychology of stereotypes and to an earlier economics literature on statistical\ndiscrimination.\n