**Assignment #3 - Cats Generator Playground**

Student name: Daniel Zeng (dlzeng)

(##) Overview

The main goal for this assignment is to become familiar with implementing and training GANs. We will do this by implementing and training a DCGAN to generate cat images from random noise and by implementing and training a CycleGAN to convert between images of different types of objects (cats in this instance).

(##) DCGAN

The idea behind the DCGAN is to use a CNN for the discriminator and to use upsampling and another CNN for the generator. 

For the Discriminator CNN, we follow the steps outlined in the writeup. For padding, we use the following formula outlined by the `conv2d` webpage:
<figure>
    <img src="writeup/padding.png" width=90% alt="padding">
</figure>

Since we are given that kernel size is 4 and stride is 2, we can substitute these values in to try to find the padding size. Our layers are decreasing the height and width by a factor of 2 at each step, so substituting this in, we have the following:

$$\frac{H}{2} = \frac{H + 2P - 1 \cdot (K - 1) - 1}{S} + 1$$
$$\implies \frac{H}{2} = \frac{H + 2P - 3 - 1}{2} + 1$$
$$\implies \frac{H}{2} = \frac{H}{2} + P - 1$$
$$\implies P = 1$$

The steps for the Generator and training loop were followed from the writeup. For the Generator, I used a convolutional layer for the first layer with kernel 4, stride 1 and padding 3. For the rest of the layers, I used `up_conv` layers with kernel 3, stride 1, padding 1 and scale factor 2.

(###) Loss curves

(####) Basic without Diffaug

<figure>
    <img src="writeup/vanilla_basic_d_loss.png" width=90% alt="DCGAN D Loss">
    <img src="writeup/vanilla_basic_g_loss.png" width=90% alt="DCGAN G Loss">
</figure>

(####) Basic with Diffaug

<figure>
    <img src="writeup/vanilla_basic_diffaug_d_loss.png" width=90% alt="DCGAN D Loss">
    <img src="writeup/vanilla_basic_diffaug_g_loss.png" width=90% alt="DCGAN G Loss">
</figure>

(####) Deluxe without Diffaug

<figure>
    <img src="writeup/vanilla_deluxe_d_loss.png" width=90% alt="DCGAN D Loss">
    <img src="writeup/vanilla_deluxe_g_loss.png" width=90% alt="DCGAN G Loss">
</figure>

(####) Deluxe with Diffaug

<figure>
    <img src="writeup/vanilla_deluxe_diffaug_d_loss.png" width=90% alt="DCGAN D Loss">
    <img src="writeup/vanilla_deluxe_diffaug_g_loss.png" width=90% alt="DCGAN G Loss">
</figure>

In the scenario the DCGAN manages to train, we would expect that the Discriminator loss and Generator loss should both slowly decrease and converge over time and not oscillate too much. We see that the Discriminator losses generally decrease over time, while the Generator losses oscillate without changing too much from it's average value.

(###) Results

For the Deluxe with Diffaug, we have the following comparison (left is at 200 iterations, right is at 6400 iterations):

<figure>
    <img src="writeup/deluxe_diffaug-000200.png" width=45% alt="DCGAN D Loss">
    <img src="writeup/deluxe_diffaug-006400.png" width=45% alt="DCGAN G Loss">
</figure>

As we can see, the GAN starts off with very blurry results, capturing the general shape of the cat face and not much else. As it trains, we can start to see the eyes, whiskers and general face shape becoming more refined.

We also want to compare this with Deluxe without Diffaug (left is at 200 iterations, right is at 6400 iterations):

<figure>
    <img src="writeup/deluxe-000200.png" width=45% alt="DCGAN D Loss">
    <img src="writeup/deluxe-006400.png" width=45% alt="DCGAN D Loss">
</figure>

Comparing the results at the end of the training, it looks like the results without Diffaug have a couple more patches of noise compared to the results with Diffaug.


(##) CycleGAN

The architecture and implementation of the CycleGAN is covered in the writeup - we simply followed the steps outlined there. 

(###) Cat Dataset

(####) Patch Discriminator without Cycle Consistency Loss

After 1000 iterations:

<figure>
    <img src="writeup/patch_no_cycle-001000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/patch_no_cycle-001000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>

(####) Patch Discriminator with Cycle Consistency Loss

After 1000 iterations:

<figure>
    <img src="writeup/patch_cycle-001000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/patch_cycle-001000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>

After 10000 iterations:

<figure>
    <img src="writeup/patch_cycle-010000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/patch_cycle-010000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>

(####) DCDiscriminator with Cycle Consistency Loss

After 1000 iterations:

<figure>
    <img src="writeup/dc_cycle-001000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/dc_cycle-001000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>

(###) Apples and Oranges Dataset

(####) Patch Discriminator without Cycle Consistency Loss

After 1000 iterations:

<figure>
    <img src="writeup/oj_patch_no_cycle-001000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/oj_patch_no_cycle-001000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>

(####) Patch Discriminator with Cycle Consistency Loss

After 1000 iterations:

<figure>
    <img src="writeup/oj_patch_cycle-001000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/oj_patch_cycle-001000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>

(####) DCDiscriminator with Cycle Consistency Loss

After 1000 iterations:

<figure>
    <img src="writeup/oj_dc_cycle-001000-X-Y.png" width=90% alt="DCGAN D Loss">
</figure>
<figure>
    <img src="writeup/oj_dc_cycle-001000-Y-X.png" width=90% alt="DCGAN D Loss">
</figure>


(###) With and Without Cycle loss

As seen in the apple and oranges dataset, there is definitely an improvement with using cycle loss over not using cycle loss. The images generated with cycle loss has more color and structure retained (colors in the first apples to orange are very dark compared to what should be expected).

(###) Patch Discriminator vs DCDiscriminator

There seems to be a difference when it comes to the background. If we take a look at the apple to oranges for DCDiscriminator, the images with background with leaves all end up getting colored orange. The patch discriminator takes a look at a patch of pixel compared to one pixel with the DCDiscriminator, so it gives more information about images around the pixel, which would help with the background colors.

(##) Bells and Whistles

I did not implement any of the Bells and Whistles.

(##) Credits

Credit to [15-468](http://graphics.cs.cmu.edu/courses/15-468/) for the HTML template.

<!--- Markdeep & image comparison library - probably no need to change anything below -->
<style class="fallback">body{visibility:hidden;white-space:pre;font-family:monospace}</style><script src="./resources/markdeep.min.js"></script><script>window.alreadyProcessedMarkdeep||(document.body.style.visibility="visible")</script>