Accuracy Improvement in Differentially Private Logistic Regression: A Pre-trainingApproach

Machine learning (ML) models can memorize trainingdatasets. As a result, training ML models on privatedatasets can lead to the violation of individuals’privacy. Differential privacy (DP) is a rigorous privacynotion to preserve the privacy of the underlyingtraining datasets. However, training ML models in aDP framework usually degrades the accuracy of MLmodels. This paper aims to increase the accuracy of aDP logistic regression (LR) via a pre-training module.In more detail, we initially pre-train our LR model on apublic training dataset without any privacy concern.Then, we fine-tune our DP-LR model with the privatedataset. In the numerical results, we show that adding apre-training module significantly improves theaccuracy of the DP-LR model.

Keywords:

Data Privacy , Differential Privacy , Trustworthy Machine Learning , Logistic Regression , Pre-training