Data-centric defense: Shaping loss landscape with augmentations to counter model inversion

Machine Learning models have shown susceptibility to various privacy attacks, with model inversion (MI) attacks posing a significant threat. Current defense techniques are mostly model-centric, involving modifying model training or inference. However, these approaches require model trainers’ cooperation, are computationally expensive, and often result in a significant privacy-utility tradeoff. To address these limitations, we propose a novel data-centric approach to mitigate MI attacks. We introduce privacy-focused data augmentations that shape the resulting model’s loss landscape, making it challenging for attackers to generate private target samples. We provide theoretical analysis explaining why such augmentations can reduce MI risk and demonstrate effectiveness and robustness across models and datasets. On face recognition benchmarks, we reduce reconstruction success rates to ≤ 5% with only ~2% accuracy drop, surpassing model-centric defenses.