Faculty, Staff and Student Publications
Language
English
Publication Date
10-1-2024
Journal
Conference: Computer and Communications Security
DOI
10.1145/3658644.3690279
PMID
40401199
PMCID
PMC12094715
PubMedCentral® Posted Date
5-21-2025
PubMedCentral® Full Text Version
Author MSS
Abstract
The pre-training and fine-tuning paradigm has demonstrated its effectiveness and has become the standard approach for tailoring language models to various tasks. Currently, community-based platforms offer easy access to various pre-trained models, as anyone can publish without strict validation processes. However, a released pre-trained model can be a privacy trap for fine-tuning datasets if it is carefully designed. In this work, we propose PreCurious framework to reveal the new attack surface where the attacker releases the pre-trained model and gets a black-box access to the final fine-tuned model. PreCurious aims to escalate the general privacy risk of both membership inference and data extraction on the fine-tuning dataset. The key intuition behind PreCurious is to manipulate the memorization stage of the pre-trained model and guide fine-tuning with a seemingly legitimate configuration. While empirical and theoretical evidence suggests that parameter-efficient and differentially private fine-tuning techniques can defend against privacy attacks on a fine-tuned model, PreCurious demonstrates the possibility of breaking up this invulnerability in a stealthy manner compared to fine-tuning on a benign pre-trained model. While DP provides some mitigation for membership inference attack, by further leveraging a sanitized dataset, PreCurious demonstrates potential vulnerabilities for targeted data extraction even under differentially private tuning with a strict privacy budget e.g. 𝜖 =0.05. Thus, PreCurious raises warnings for users on the potential risks of downloading pre-trained models from unknown sources, relying solely on tutorials or common-sense defenses, and releasing sanitized datasets even after perfect scrubbing.
Keywords
Privacy Attack, Language Model, Pre-Training
Published Open-Access
yes
Recommended Citation
Liu, Ruixuan; Wang, Tianhao; Cao, Yang; et al., "PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps" (2024). Faculty, Staff and Student Publications. 926.
https://digitalcommons.library.tmc.edu/uthshis_docs/926