2024/10/09 by Jinzhe Tan, Hannes Westermann, Tan, Jinzhe +11 · 4 citations
Business, Management and Accounting · Computer Science · Social Sciences · #Artificial Intelligence in Law #Computation and Language (cs.CL) #Dispute Resolution and Class Actions #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Law, AI, and Intellectual Property
paper · pdf · doi:10.48550/arxiv.2410.07053
openalex publication_date 2024/10/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Mediation is a dispute resolution method featuring a neutral third-party (mediator) who intervenes to help the individuals resolve their dispute. In this paper, we investigate to which extent large language models (LLMs) are able to act as mediators. We investigate whether LLMs are able to analyze dispute conversations, select suitable intervention types, and generate appropriate intervention messages. Using a novel, manually created dataset of 50 dispute scenarios, we conduct a blind evaluation comparing LLMs with human annotators across several key metrics. Overall, the LLMs showed strong performance, even outperforming our human annotators across dimensions. Specifically, in 62% of the cases, the LLMs chose intervention types that were rated as better than or equivalent to those chosen by humans. Moreover, in 84% of the cases, the intervention messages generated by the LLMs were rated as better than or equal to the intervention messages written by humans. LLMs likewise performed favourably on metrics such as impartiality, understanding and contextualization. Our results demonstrate the potential of integrating AI in online dispute resolution (ODR) platforms.