Research Project: Building a Generative Adversarial Network

Image for post
Image for post
Face generated by a GAN

Goals and Overview

A Generative Adversarial Network (GAN) is a type of neural network for generative modeling. The generative model is trained using two neural network models: a generator and a discriminator. GANs are complex so to put them in laymen’s terms, the way they work is there is a generator model that learns to generate plausible samples from a dataset. There is also a discriminator set that is provided with the same dataset and it’s goal is to distinguish real data from fake data (data that is produced by the generator model). Both models enter a game, so to speak, where the generator seeks to fool the discriminator by learning and generating data that is so similar to the shared dataset that the discriminator is unable to distinguish generated data from actual data. This game that the models play is where the word “Adversarial” comes from in the network name.

In this research project, I sought to learn about and become more familiar with GANs, and, in particular, image producing GANs. Because GANs are very computationally heavy, the MNIST dataset was selected for because it is a smaller, simpler dataset and thus less computationally taxing.

Practical Application of GANs

  • GANs are practical tools to generate plausible data from a dataset which allows those working on the dataset to have more data to work with.
  • GANs can be used to enhance security software: by creating more types of attacks on a security network, the network can adapt to these innocuous attacks preparing it for new plausible attacks in the future.
  • GANs can be used for Image Editing, giving artists and graphic designers new ways to modify their work as showcased below:
  • GANs can be used for generating images from a learned dataset. Generating new images was the scope of my research project.

Terminology

When training the model, it is not usually possible to feed all of the data into the algorithm at one time because of the memory limitations and data sizes so the data must be broken into smaller pieces. This was certainly the case for my project even though I purposely chose a simple and small dataset to work with. Here are two key terms that were important to understanding how GANs work:

  • Epoch: An epoch is a measure of when a dataset has been passed forward and backward through the neural network exactly one time.
  • Batch Size: Because datasets are too large to pass through the network all at once, it needs to be broken up into smaller samples of the dataset known as Batches. The Batch Size is the number of samples in each Batch.
  • Iteration: An Iteration is the number of batches that must be completed in order to complete one Epoch. So if you have a dataset with 1000 images that is split into Batches of 100 images, it will require 10 iterations to complete one epoch.

Generated Images

Here is an example of some of the digits that the discriminator model was trained on.

The image above shows digits that have been produced by the generator model after 10 epochs. At this point, the generator has learned from the discriminator where to centralize crude digits.

After 50 epochs, the digits have gained more “curve” and begin to look more like digits.

After 100 epochs, even more smoothening has happened and the figures are resembling more and more like actual digits.

Future Work

For my future work, I would like to create a GAN to generate faces. The added complexity of the images will pose new challenges with which to improve my knowledge of GANs. GANs are very complicated so I’m aiming to learn more about how they work on the backend. I hope to use this new technology in my field of work if ever it is applicable.

Here is the link to my work on GitHub:

https://github.com/jkundycki/Capstone/tree/master/MNIST

Thanks for reading and best of luck to you,

John

Written by

Data Scientist | Health Enthusiast | Learner

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store