privacy

Do Membership Inference Attacks Work on Large Language Models?
A large-scale evaluation of membership inference attacks (MIAs) on LLMs shows that MIAs perform barely better than random guessing, attributed to large datasets, few training iterations, and fuzzy boundaries between data members.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, Hannaneh Hajishirzi
PDF Cite Code Dataset Project
SoK: Memorization in General-Purpose Large Language Models
We explore the memorization capabilities of Large Language Models (LLMs), categorizing them into six types, and discuss their implications and challenges.
Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, Robert West
PDF Cite
SoK: Let The Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning
An SoK that presents a game-based framework to systematize the body of knowledge on privacy inference risks in machine learning.
Ahmed Salem, Giovanni Cherubin, David Evans, Boris Köpf, Andrew Paverd, Anshuman Suri, Shruti Tople, Santiago Zanella-Béguelin
PDF Cite Video