Deep Convolutional GAN (DCGAN). A DCGAN is simply a GAN that uses a convolutional neural network as the discriminator, and a network composed of transposed convolutions as the generator.
In the assignment, instead of using transposed convolutions, we will be using a combination of a upsampling layer and a convoluation layer to replace transposed convolutions.
Padding: In each of the convolutional layers shown above, we downsample the spatial dimension of the input volume by a factor of 2. Given that we use kernel size K = 4 and stride S = 2, what should the padding be? Write your answer on your website, and show your work (e.g., the formula you used to derive the padding).
Solving with W out = 32, W = 64, we get padding = 1
Instead of an up sampling layer, my first layer is a conv layer with kernel size 4, step1, padding 3, which also upsamples the 1x1 input into a 4x4 output.
The results below demonstrate that:
1. Deluxe data preprocessing significantly decreased generator and descriminator loss.
2. Incorporating differentiable augmentation to reduce overfitting also demonstrated a visible decrease in descriminator losses in both basic and deluxe experiments.
3. The loss of discriminator and generator fluctuates around 1 in the basic case, and the discriminator has slightly lower loss in the deluxe case.
If the GAN manages to train, the losses of the generator and discriminator should both fluctuate and neither of them should win significantly over the other.
Basic Generator loss
Basic Discriminator loss
Basic + diffaug Generator loss
Basic + diffaug Discriminator loss
Deluxe Generator loss
Deluxe Discriminator loss
Deluxe + diffaug Generator loss
Deluxe + diffaug Discriminator loss
Towards the beginning of training, at 200 epochs, the samples are blurry, with rectangular color regions that look unatural. At the end of the traning process, at 6400 epochs, the images are more realistic with smoother edges between different colors.
Deluxe + diffaug iter 200
Deluxe + diffaug iter 6400
Without cycle-consistency loss
python cycle_gan.py --disc patch --train_iters 1000
iter 1000, X-Y
iter 1000, Y-X
Wth cycle-consistency loss
python cycle_gan.py --disc patch --use_cycle_consistency_loss --train_iters 1000
iter 1000, X-Y
iter 1000, Y-X
Without cycle-consistency loss 10000 iters
python cycle_gan.py --disc patch --train_iters 10000
iter 10000, X-Y
iter 10000, Y-X
With cycle-consistency loss 10000 iters
python cycle_gan.py --disc patch --use_cycle_consistency_loss --train_iters 10000
iter 10000, X-Y
iter 10000, Y-X
Without cycle-consistency loss 1000 iters
python cycle_gan.py --disc patch --train_iters 1000 --X apple2orange/apple --Y apple2orange/orange
iter 1000, X-Y
iter 1000, Y-X
With cycle-consistency loss 1000 iters
python cycle_gan.py --disc patch --use_cycle_consistency_loss --train_iters 1000 --X apple2orange/apple --Y apple2orange/orange
iter 1000, X-Y
iter 1000, Y-X
Without cycle-consistency loss 10000 iters
python cycle_gan.py --disc patch --train_iters 10000 --X apple2orange/apple --Y apple2orange/orange
iter 10000, X-Y
iter 10000, Y-X
With cycle-consistency loss 10000 iters
python cycle_gan.py --disc patch --use_cycle_consistency_loss --train_iters 10000 --X apple2orange/apple --Y apple2orange/orange
iter 10000, X-Y
iter 10000, Y-X
1. With cycle-consistency loss, the original details are better recovered. This is because we set the loss to also minimize the MSE between original images and the imags transformed from the original and then back. The generative model is not totally focused on tricking the discriminator but also on recovering the original image features. For example, without cycle-consistency loss, small details such as the cat's eyes might not be there in the reconstructed image.
2. However, without cycle-consistency loss, the generated image can look more realistic to human eyes. This is because the model doesn't have to merge features from images of two objects that don't look alike and lose fome naturalness in the image originally generated. For example, in the case of the apple on the top-left corner.
CycleGAN + DCDiscriminator with cycle-consistency loss 10000 iters
python3 cycle_gan.py --disc dc --use_cycle_consistency_loss --train_iters 10000
iter 10000, X-Y
iter 10000, Y-X
With DCDiscriminator, we lose some details that are present in the case of patch discriiminator, as DCDiscriminator focuses more globally than locally on finer details. For example, the cats' eyes are especially blurry, have sharp, unatural boundaries, and distorted.
python train_ddpm.py
python test_ddpm.py
Some samples:
