Assignment 3: Cats Generator Playground

Student name: Tianxiang Lin
Andrew ID: tianxian

Project Overview

This project aims at implementing Deep Convolution GAN (DCGAN) and CycleGAN for image generation and image-to-image style transformation.

Deep Convolution GAN (DCGAN)

Padding

In each of the convolutional layers shown above, we downsample the spatial dimension of the input volume by a factor of 2. Given that we use kernel size K = 4 and stride S = 2, what should the padding be? Write your answer on your website, and show your work (e.g., the formula you used to derive the padding).

$$\text{output} = \lfloor (\text{input} + 2 * p - k) / s \rfloor + 1.$$

Given that k=4, s=2, input/output=2 (downsampling),

$$p=1.$$

Differentiable Augmentation

In the write up, please show results with and without applying differentiable augmentations, and discuss the difference between two augmentation schemes we discussed, in terms of implementation and effects.

The following sheet demonstrates the results with and without differentiable augmentations and other customized data augmentations. From the results, adding data augmentations will increase the quality of generated images in different aspects.

While other data augmentation techniques soften the colors of images compared to basic data augmentations, differentiable augmentations generate much smoother images with less artifacts in the resulting images. This is because differentiable augmentation does better to prevent overfitting during training.

Basic	Basic_diffaug	Deluxe	Deluxe_diffaug

Experiments

Screenshots of discriminator and generator training loss with --data_preprocess=basic, --data_preprocess=deluxe. For each --data_preprocess flag, also show results trained both with and without differentiable augmentation, so you will show 8 curves in total. Briefly explain what the curves should look like if GAN manages to train.

The following sheet shows the training loss by time when applying different training strategies. Each plot contains two curves that are from with Augdiff and without.

When managing to train a GAN, from the observations from above the curves for both discriminator and generator should fluctuate around a small loss with low covariance.

Method	Plot
Basic (Disc)
Basic (Gen)
Deluxe (Disc)
Deluxe (Gen)

With --data_preprocess=deluxe and differentiable augmentation enabled, show one of the samples from early in training (e.g., iteration 200) and one of the samples from later in training, and give the iteration number for those samples. Briefly comment on the quality of the samples, and in what way they improve through training.

They quality of the samples increase with more training epochs. The model always generates blurry cat-sharp initially and with time it starts to create clear and seamless cat images.

Deluxe_diffaug_200	Deluxe_diffaug_6400

CycleGAN

Experiments

Include the sampled output from your model.

Dataset	X to Y	Y to X
cat, 1000 iters (w/o cycle consistency+patch)
cat, 1000 iters (w/ cycle consistency+patch)
cat, 10000 iters (w/o cycle consistency+patch)
cat, 10000 iters (w/ cycle consistency+patch)
cat, 10000 iters (w/ cycle consistency+dc)
apple2orange, 1000 iters (w/o cycle consistency+patch)
apple2orange, 1000 iters (w/ cycle consistency+patch)
apple2orange, 10000 iters (w/o cycle consistency+patch)
apple2orange, 10000 iters (w/ cycle consistency+patch)
apple2orange, 10000 iters (w/ cycle consistency+patch)

Do you notice a difference between the results with and without the cycle consistency loss? Write down your observations (positive or negative) in your website. Can you explain these results, i.e., why there is or isn’t a difference between the two?

Results with cycle consistency loss are much better than ones without it. They are much smoother and with less artifacts, while ones without cycle consistency loss are blurry. This is because cycle consistency loss helps the generator reconcile the inconsistency during stochastic training process, especially with two generative models.

Compare and report your observations between the results using DCDiscriminator and PatchDiscriminator. Can you explain the results?

Patch Discriminator creates better results than DC Discriminator. It generates smoother and clearer results, especially for crucial features like cat's eyes, while DC Discriminator messes them up. This is because Patch Discriminator takes spatiality into account. The model discerns local structures better and gives better loss to be back propagated.