Welcome to the webpage for Kevin You's submission for project 3.
We start by implementing a DCGAN operating on 64x64 images with least squares loss. The specific architecture is described in the assignments page. This will allow us to generate images of grumpy cats. For data augmentation I did a random crop with size 1.1x and a random horizontal flip with p=0.2. The choice of constants is mostly arbitrary, but it worked well enough.
In the discriminator we wish to downsaple the spatial dimensions by a factor of 2, with kernel size K = 4 and stride S = 2. A stride of 2 downsample by roughly a factor of 2, and we need to add padding so that this factor is exactly 2. Supppose that we ended up with width N. Then, we must have started with width SN + (K-S) = 2N + 2. (Technically we could have started with up to 3 pixels more, and not use the last few pixels, but why would we do that?) Thus, we should have a padding 1 on both sides. For this type of calculation, it becomes clear that by linearity, N does not matter.
| Basic | Deluxe | Basic Diff | Deluxe Diff | |
|---|---|---|---|---|
| D | |
|
|
|
| G | |
|
|
|
Shown above is the lost curves. For GAN's, the loss curve doesn't mean much. Even if the generator trains, the discriminator could also get better, which increases the generator's loss. We do see that without augmention (basic), the discriminator's loss is too small, suggesting potential overfitting. Augmentation did help prevent this.
| Basic | Deluxe | Basic Diff | Deluxe Diff | |
|---|---|---|---|---|
| Iteration 400 |  |
 |
 |
 |
| Iteration 6400 |  |
 |
 |
 |
The samples confirm our finding that data augmention (deluxe versus basic) helps produce more diverse samples. The differentiable augmentation also seemed to help slightly, as the best samples seemed to come from deluxe diff (e.g. top right). Obviously the samples are pretty bad at 400 iterations, since neither the discriminator nor the generator has trained enough. Only the colors are vaguely correct (luckily since our dataset contains fairly similar images).
Next we implement a cycleGAN to convert between two types of cats or apples and oranges. The architecture pretty closely follws the DCGan part where we pretty much keep the first part of the discriminator and second part of the generator, with some extra residual blocks in-between.
| Cat | Cat Cycle | Apple | Apple Cycle | |
|---|---|---|---|---|
| Iteration 1000, X to Y |  |
 |
 |
 |
| Iteration 10000, X to Y |  |
 |
 |
 |
| Iteration 1000, Y to X |  |
 |
 |
 |
| Iteration 10000, Y to X |  |
 |
 |
 |
First to note is that training more iterations helped get better pictures, especially for without the cycle consistency loss. With cycle consistenvy loss, training seemed much faster. With and without cycle consistency, the results are very different, as emphasized in the cats example. With no cycle consistency (say X to Y), the results look actually like cats of Y, with a slightly resemblence to the input X. With cycle consistency, the results look much more like cats of X, with some colors from Y. Which of these is desirable is of course dependent of the user, though usually users will want cycle consistency.
| Cat | Cat Cycle | Apple | Apple Cycle | |
|---|---|---|---|---|
| Iteration 9900, X to Y, DC |  |
 |
 |
 |
| Iteration 9900, Y to X, DC |  |
 |
 |
 |
Finally we look at changing the patch based discriminator to the DC discriminator. The cats results look pretty similar, but between the fruits, in the case where there are multiple fruits in an image, it seems like the patch discriminator was better, since each fruit could be treated seperately.