GAN Tutorial - From basics to current state-of-the-art, and towards key applications in medicine

Abstract

In generative modelling, Generative Adversarial Networks (GANs) have recently obtained state-of-the-art results on a variety of image generation tasks. However, it is not always clear why such generative models are useful. Why would one be interested to generate the (fake) brain of a person that doesn’t exist? Or a fake chest X-ray? In this tutorial-talk, I will give a brief primer on GAN models, then discuss recent state-of-the-art models (incl. NVIDIA’s StyleGAN2 & Limited-data GAN) as well as key applications in medical image reconstruction: super-resolution, inpainting, slice-imputation, motion-correction as well as MRI reconstruction of undersampled k-space. Finally, I will present preliminary results using the same state-of-the-art StyleGAN2 model on three widely different datasets: Chest X-rays (MIMIC III), T2 brain scans (ADNI/OASIS/AIBL/PPMI/…), and microscopy images of pancreatic cancer (MICCAI PANDAS 2020 dataset). I will also discuss the potential application of GANs to predicting future disease progression, a key clinical problem in today’s healthcare.