2024/01/10 by Sayash Kapoor, Peter Henderson, Kapoor, Sayash +3 · 4 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Law #Computers and Society (cs.CY) #FOS: Computer and information sciences #Law, AI, and Intellectual Property
paper · pdf · doi:10.48550/arxiv.2402.01656
openalex publication_date 2024/01/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Is AI set to redefine the legal profession? We argue that this claim is not supported by the current evidence. We dive into AI's increasingly prevalent roles in three types of legal tasks: information processing; tasks involving creativity, reasoning, or judgment; and predictions about the future. We find that the ease of evaluating legal applications varies greatly across legal tasks, based on the ease of identifying correct answers and the observability of information relevant to the task at hand. Tasks that would lead to the most significant changes to the legal profession are also the ones most prone to overoptimism about AI capabilities, as they are harder to evaluate. We make recommendations for better evaluation and deployment of AI in legal contexts.