Classifying complex documents: comparing bespoke solutions to large language models

2023/12/12 by Glen Hopkins, Hopkins, Glen, Kristjan Kalm +1
Social Sciences · #Artificial Intelligence in Law #Computational and Text Analysis Methods

paper · pdf · doi:10.48550/arxiv.2312.07182

Abstract

Here we search for the best automated classification approach for a set of complex legal documents. Our classification task is not trivial: our aim is to classify ca 30,000 public courthouse records from 12 states and 267 counties at two different levels using nine sub-categories. Specifically, we investigated whether a fine-tuned large language model (LLM) can achieve the accuracy of a bespoke custom-trained model, and what is the amount of fine-tuning necessary.

Related

• Exploiting the Randomness of Large Language Models (LLM) in Text Classification Tasks: Locating Privileged Documents in Legal Matters
• Large Language Models for Judicial Entity Extraction: A Comparative Study
• Long-length Legal Document Classification
• One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial Support
• Exploring Large Language Models and Hierarchical Frameworks for Classification of Large Unstructured Legal Documents