Privacy for free in the overparameterized regime.

ABSTRACT: Differentially private gradient descent (DP-GD) is a popular algorithm to train deep learning models with provable guarantees on the privacy of the training data. In the last decade, the problem of understanding its performance cost with respect to standard GD has received remarkable attention from the research community, which has led to upper bounds on the excess population risk [Formula: see text] in different learning settings. However, such bounds typically degrade with overparameterization, i.e., as the number of parameters [Formula: see text] gets larger than the number of training samples [Formula: see text]-a regime which is ubiquitous in current deep-learning practice. As a result, the lack of theoretical insights leaves practitioners without clear guidance, leading some to reduc

SUBMITTER: Bombari S 

PROVIDER: S-EPMC12012534 | biostudies-literature | 2025 Apr

REPOSITORIES: biostudies-literature