membership inference

Do Parameters Reveal More than Loss for Membership Inference?
We show how prior claims about black-box access sufficing for optimal membership inference do not hold for most useful settings such as SGD, and validate our findings with a new white-box inference attack.
Anshuman Suri, Xiao Zhang, David Evans
PDF Cite Code
Do Membership Inference Attacks Work on Large Language Models?
A large-scale evaluation of membership inference attacks (MIAs) on LLMs shows that MIAs perform barely better than random guessing, attributed to large datasets, few training iterations, and fuzzy boundaries between data members.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, Hannaneh Hajishirzi
PDF Cite Code Dataset Project
SoK: Memorization in General-Purpose Large Language Models
We explore the memorization capabilities of Large Language Models (LLMs), categorizing them into six types, and discuss their implications and challenges.
Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, Robert West
PDF Cite
My submission to the MICO Challenge
Description of my entry to the MICO challenge (co-located with SaTML) for membership inference that won me the 2nd place on the CIFAR track.
Anshuman Suri
Last updated on Nov 9, 2023 14 min read
My submission to the MICO Challenge