LoRA Provides Differential Privacy by Design via Random Sketching

2024/09/26 by Saber Malekmohammadi, Golnoosh Farnadi, Malekmohammadi, Saber +1 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2409.17538

openalex publication_date 2024/09/26 · openalex created_date 2024/10/27 · openalex updated_date 2026/07/29

Abstract

Low-rank adaptation of language models has been proposed to reduce the computational and memory overhead of fine-tuning pre-trained language models. LoRA incorporates trainable low-rank matrices into some parameters of the pre-trained model, called adapters. In this work, we show theoretically that the low-rank adaptation mechanism of LoRA is equivalent to fine-tuning adapters with noisy batch gradients, with the noise variance being a decreasing function of adaptation rank (r). Motivated by this understanding, we prove inherent differential privacy for LoRA when adaptation matrices A_ℓ are frozen. We show that various factors, e.g., the adaptation rank and batch size, affect the guaranteed privacy level. Our findings provide useful insights into LoRA and uncovers the reason behind the robustness of models fine-tuned with LoRA to privacy attacks.

Cited by

• How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding

Related

• Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?
• LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Models
• Sparse Low-rank Adaptation of Pre-trained Language Models
• LoRA vs Full Fine-tuning: An Illusion of Equivalence
• The Expressive Power of Low-Rank Adaptation