Analyzing Bias in Swiss Federal Supreme Court Judgments Using Facebook's Holistic Bias Dataset: Implications for Language Model Training

2025/01/06 by Sabine Wehnert, Wehnert, Sabine, Muhammet Ertas +3
Business, Management and Accounting · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Dispute Resolution and Class Actions #European and International Law Studies #FOS: Computer and information sciences #Legal Language and Interpretation

paper · pdf · doi:10.48550/arxiv.2501.03324

openalex publication_date 2025/01/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Natural Language Processing (NLP) is vital for computers to process and respond accurately to human language. However, biases in training data can introduce unfairness, especially in predicting legal judgment. This study focuses on analyzing biases within the Swiss Judgment Prediction Dataset (SJP-Dataset). Our aim is to ensure unbiased factual descriptions essential for fair decision making by NLP models in legal contexts. We analyze the dataset using social bias descriptors from the Holistic Bias dataset and employ advanced NLP techniques, including attention visualization, to explore the impact of dispreferred descriptors on model predictions. The study identifies biases and examines their influence on model behavior. Challenges include dataset imbalance and token limits affecting model performance.

Related

• Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Dataset
• Questioning Biases in Case Judgment Summaries: Legal Datasets or Large Language Models?
• A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges
• Bias in Large Language Models: Origin, Evaluation, and Mitigation
• Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction\n Benchmark