Training a generative model on medical images, measuring whether it leaks its training data, and adding differential privacy to bound that leakage. Built on PneumoniaMNIST (chest X-rays, 28x28) as a laptop-scale study of the privacy versus utility trade-off in synthetic medical data.
This repository accompanies an application to the Linköping University WASP PhD position on generative modeling for data-efficient machine learning. It is an honest in-progress study, not a finished paper, and the limitations below are stated plainly.
- A small DCGAN can generate recognisable synthetic chest X-rays.
- The synthetic images carry real signal: a classifier trained only on them reaches most of the accuracy of one trained on real data.
- A simple membership-inference attack finds no detectable leakage in the plain generator, but a weak attack failing is not a guarantee of safety.
- Differential privacy (DP-SGD) provides a formal guarantee that holds against any attack, at a measurable cost to accuracy and image quality.
The DCGAN was trained for 40 epochs on the MPS backend. Real reference
samples are in figures/real_samples.png; generated samples in
figures/gan_samples.png. The generated grid shows clear ribcages and
lung fields, with variation across tiles indicating no mode collapse.
| Metric | Value |
|---|---|
| FID (lower is better) | 85.75 |
| Accuracy, train on synthetic, test on real (TSTR) | 0.846 |
| Reference accuracy, train on real, test on real | 0.894 |
TSTR sitting close to the real-data reference, and well above the majority- class rate of about 0.63, shows the synthetic images are genuinely useful. The FID value is high in absolute terms, as expected for a small GAN on 28x28 images, but it is a consistent yardstick that responds to quality changes, which is what matters for the privacy comparison.
A nearest-neighbour distance attack asks whether a real image was in the training set. The attack AUC was 0.480, essentially chance. Mean nearest- synthetic distance was 5.265 for training members and 5.295 for held-out non-members, an overlap the histogram makes visually clear. The plain GAN shows no leakage detectable by this attack.
DP-SGD was applied to the classifier at three privacy budgets. Accuracy falls monotonically as the budget tightens, the expected privacy-utility trade-off.
| Privacy budget (epsilon) | Test accuracy |
|---|---|
| Non-private baseline | 0.811 |
| 10 | 0.779 |
| 3 | 0.755 |
| 1 | 0.705 |
As an extension, the DCGAN itself was retrained with DP-SGD applied to the
discriminator, the only component that touches real data; the generator
inherits the guarantee by the post-processing property of differential
privacy. Training reached an achieved epsilon of 9.99 against a target of 10.
The private samples in figures/gan_samples_dp.png remain recognisable as
chest X-rays but are visibly softer and grainier than the non-private grid,
a direct picture of the cost of the guarantee.
data.py load and normalise MedMNIST to [-1, 1] gan.py DCGAN generator and discriminator model.py small ConvNet classifier train_gan.py Phase 1: train the generator quality.py Phase 2: FID and TSTR mia.py Phase 3: membership-inference attack train_dp_classifier.py Phase 4: DP-SGD privacy-utility trade-off train_gan_dp.py Phase 4 bonus: differentially private DCGAN tradeoff.py Phase 5: privacy-utility plot figures/ saved plots and sample grids results/ CSV outputs
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python train_gan.py # train the generator
python quality.py # FID + TSTR
python mia.py # membership-inference attack
python train_dp_classifier.py # DP-SGD trade-off (runs on CPU)
python tradeoff.py # privacy-utility plot
python train_gan_dp.py # DP-GAN bonus (runs on CPU)Differentially private steps run on CPU because Opacus is unstable on the MPS backend. Everything else uses MPS where available.
This is a small, honest study and its scope is limited in several ways.
The generator is unconditional, so it does not know each image's class. For the TSTR evaluation, synthetic images were labelled by a classifier trained on real data; a conditional GAN would remove this step and is the natural next direction.
The membership-inference attack is a simple nearest-neighbour test. It found no leakage, but a stronger attack, such as a shadow-model or LOGAN-style attack, might. The value of the differential privacy phase is that it gives a formal guarantee that does not depend on which attack is tried.
The full three-budget privacy sweep was run on the classifier, which is the standard and reliable DP-SGD setting. The DP-GAN was run at a single budget as a bonus. Extending the DP-GAN across several budgets, and recomputing FID and attack AUC under each, is the clear next step.
Everything is laptop-scale: 28x28 images, a small DCGAN, and short training runs. The class distribution is also imbalanced, with roughly three times as many pneumonia as normal cases, which a generative model will tend to inherit.
Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K. and Zhang, L., 2016. Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, pp.308-318.
Hayes, J., Melis, L., Danezis, G. and De Cristofaro, E., 2019. LOGAN: membership inference attacks against generative models. Proceedings on Privacy Enhancing Technologies, 2019(1), pp.133-152.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. and Hochreiter, S., 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Advances in Neural Information Processing Systems 30 (NeurIPS 2017), pp.6626-6637.
Radford, A., Metz, L. and Chintala, S., 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In: International Conference on Learning Representations (ICLR 2016).
Shokri, R., Stronati, M., Song, C. and Shmatikov, V., 2017. Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP). IEEE, pp.3-18.
Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H. and Ni, B., 2023. MedMNIST v2: a large-scale lightweight benchmark for 2D and 3D biomedical image classification. Scientific Data, 10(1), 41.
Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj, A., Zhao, J., Cormode, G. and Mironov, I., 2021. Opacus: user-friendly differential privacy library in PyTorch. arXiv preprint arXiv:2109.12298.