Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning

2024/09/06 by Xinyue Liu, Liu, Xinyue, Harshita Diddee +3 · 5 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2409.04574

openalex publication_date 2024/09/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

One-size-fits-all large language models (LLMs) are increasingly being used to help people with their writing. However, the style these models are trained to write in may not suit all users or use cases. LLMs would be more useful as writing assistants if their idiolect could be customized to match each user. In this paper, we explore whether parameter-efficient finetuning (PEFT) with Low-Rank Adaptation can effectively guide the style of LLM generations. We use this method to customize LLaMA-2 to ten different authors and show that the generated text has lexical, syntactic, and surface alignment with the target author but struggles with content memorization. Our findings highlight the potential of PEFT to support efficient, user-level customization of LLMs.

Cited by

• JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew
• One Agent to Serve All: a Lite-Adaptive Stylized AI Assistant for Millions of Multi-Style Official Accounts
• Catch Me If You Can? Not Yet: LLMs Still Struggle to Imitate the Implicit Writing Styles of Everyday Authors
• ChatGPT-generated texts show authorship traits that identify them as non-human
• StyleAdaptedLM: Enhancing Instruction Following Models with Efficient Stylistic Transfer

Related

• Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper
• An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models
• Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs
• Can Smaller LLMs do better? Unlocking Cross-Domain Potential through Parameter-Efficient Fine-Tuning for Text Summarization
• Are LLMs Effective Backbones for Fine-tuning? An Experimental Investigation of Supervised LLMs on Chinese Short Text Matching