Assignment #3 - Cats Generator Playground

1. Project Overview

In this project, we implement the model architecture and training loop for DCGAN and CycleGAN. Additionally, we perform ablation study and analysis on data augmentation method and discriminator type to provide further information.

2. DCGAN

2.1 Padding size for discriminator

The formula for calculating the relationship between input size, output size, and convolutional paramter is shown below:

output_size = (\frac{input_size + 2 \times padding - kernel_size}{stride}) + 1

$output\_size$ $input\_size$ $kernel\_size$ $stride$ = 2:

\begin{matrix} \frac{i n p u t_s i z e}{2} = \frac{i n p u t_s i z e + 2 \times p a d d i n g - 4}{2} + 1 \end{matrix}

$padding$ $input\_size$ and it is fixed to be 1.

2.2 Padding size for generator

$output\_size$ $input\_size$ $kernel\_size$ $stride$ $padding$ $kernel\_size$ $padding$ = 1, there would exist 0 element after convolution and make each noise approximately the same.

2.3 Loss curve for different data augmentation

(1) --data_preprocess=basic

Discriminator total loss:

grumpify_basis_D

Generator total loss:

grumpify_basis_G

(2) --data_preprocess=basic and --use_diffaug

Discriminator total loss:

grumpify_basis_diffaug_D

Generator total loss:

grumpify_basis_diffaug_G

(3) --data_preprocess=deluxe

Discriminator total loss:

grumpify_deluxe_D

Generator total loss:

grumpify_deluxe_G

(4) --data_preprocess=deluxe and --use_diffaug

Discriminator total loss:

grumpify_deluxe_diffaug_D

Generator total loss:

grumpify_deluxe_diffaug_G

Ideally, the discriminator loss would decrease at the early stage and eventually be stablized around one value that suggests around 50% accuracy in distinguishing real images from fake ones. The generator loss would be high at the early stage and decreases over time as it learns to produce high-quality images. Additionally, the loss curve would fluctuate a lot due to the dynamics due to the adversarial training.

2.4 Deluxe and diffaug GAN example

Generated image at iteration 200:

vanilla_gan_example_iter200

Generated image at iteration 2000:

vanilla_gan_example_iter2000

Generated image at iteration 6000:

vanilla_gan_example_iter6000

Based on three example from 200 to 2000 to 6000 above, we can see that GAN gradually improves its generation quality by making the edge and shape better as the first step. Based on clear edge and shape, it would improve its details including the eyes and fur.

3. CycleGAN

3.1 CycleGAN w/ and w/o cycle-consistency loss example

Cat dataset: w/o cycle consistency loss at iteration 1000

X-Y

naive_1000-X-Y

Y-X

naive_1000-Y-X

Cat dataset: w/ cycle consistency loss at iteration 1000

X-Y

with_consistency_1000-X-Y

Y-X

with_consistency_1000-Y-X

3.2 Training 10000 iterations cat

Cat dataset: w/o cycle consistency loss at iteration 10000

X-Y

no_cycle_consistency_loss-X-Y

Y-X

no_cycle_consistency_loss-X-Y

Cat dataset: w/ cycle consistency loss at iteraction 10000

X-Y

w:o cycle consistency loss at iteration 10000-X-Y

Y-X

w:o cycle consistency loss at iteration 10000-Y-X

3.3 Training 10000 iterations on apple2orange

Apple2orange dataset: w/o cycle consistency loss at iteration 10000

X-Y

no_cycle_consistency_loss-X-Y_appleorange

Y-X

no_cycle_consistency_loss-Y-X_appleorange

Apple2orange dataset: w/ cycle consistency loss at iteration 10000

X-Y

w: cycle consistency loss at iteration 10000-X-Y

Y-X

w: cycle consistency loss at iteration 10000-Y-X

3.4 Consistency loss role

The positive side is that consistency loss helps the model generate more clear edges and object shapes since it forces the model to match generated images with real images. Easy features like edges and colors are more similar with real images compared with generated images provided without consistency loss. The consistency loss helps the model avoid generating ambiguous or unclear images.

However, the current comparison are purely subjective. More advanced automatic evaluation metric like FID should be included to provide accuracy comparison.

3.5 DCDiscriminator vs PatchDiscriminator

Generally, PatchDiscriminator helps CycleGAN provide higher quality of images. The patch classification helps the discriminator to better classify between generated images and real images. Typically, patches features help the model better align with local features like the shape of eyes and the shape of months. Examples are provided below.

Cat dataset: CycleGAN with DCDiscriminator at iteration 10000

X-Y

cat_dc-X-Y