Faculty, Staff and Student Publications
Language
English
Publication Date
1-1-2025
Journal
Advances in Neural Information Processing Systems
PMID
42421992
PMCID
PMC13345486
Abstract
Large language models (LLMs) raise concerns about content authenticity and integrity because they can generate human-like text at scale. Text watermarks, which embed detectable statistical signals into generated text, offer a provable way to verify content origin. Many detection methods rely on pivotal statistics that are i.i.d. under human-written text, making goodness-of-fit (GoF) tests a natural tool for watermark detection. However, GoF tests remain largely underexplored in this setting. In this paper, we systematically evaluate eight GoF tests across three popular watermarking schemes, using three open-source LLMs, two datasets, various generation temperatures, and multiple post-editing methods. We find that general GoF tests can improve both the detection power and robustness of watermark detectors. Notably, we observe that text repetition, common in low-temperature settings, gives GoF tests a unique advantage not exploited by existing methods. Our results highlight that classic GoF tests are a simple yet powerful and underused tool for watermark detection in LLMs.
Published Open-Access
yes
Recommended Citation
He, Weiqing; Li, Xiang; Shang, Tianqi; et al., "On the Empirical Power of Goodness-of-Fit Tests in Watermark Detection" (2025). Faculty, Staff and Student Publications. 889.
https://digitalcommons.library.tmc.edu/uthshis_docs/889