_________________________________________________________

HW3 - Cats Generator Playground

Experiment with DCGANs [50 points]

EXP 1 -- training loss

The provided figures illustrate the training loss across four configurations for our initial task.

basic	basic diffaug
deluxe	deluxe diffaug

If the GAN trains effectively, we should observe a specific trend: the generator's loss decreases as it becomes more adept at producing samples closely resembling real ones. Simultaneously, the discriminator's loss diminishes as it improves its ability to differentiate between genuine and fake samples crafted by the generator. Ideally, both losses will reach a stable plateau, signifying that the generator is capable of creating high-quality samples that the discriminator struggles to identify as fake. This equilibrium state indicates the GAN's success in generating samples of commendable quality.

EXP 2 -- training results

With --data_preprocess=deluxe and differentiable augmentation enabled, two training samples below:

iter = 400	iter = 6000

Initially, the images generated might be of low quality, appearing blurry or marred by artifacts. However, with prolonged training, there is a noticeable improvement in the quality of the generated images. They become sharper, more detailed, and increasingly resemble the real samples, closely mirroring the characteristics of the training dataset.

CycleGAN Experiments [50 points]

EXP 1 -- cat with/without cycle_consistency_loss 1000 iters

samples are shown below

iter = 1000 w/o cycle_consistency_loss (X->Y)	iter = 1000 w/o cycle_consistency_loss (Y->X)
iter = 1000 with cycle_consistency_loss (X->Y)	iter = 1000 with cycle_consistency_loss (Y->X)

EXP 2 -- with cycle_consistency_loss 10000 iters

The results for 10000 iterations are below:

iter = 10000 (X->Y)	iter = 10000 (Y->X)

EXP 3 -- apple2organge cycle_consistency_loss test

iter = 10000 Patch w/o cycle_consistency_loss (X->Y)	iter = 10000 Patch w/o cycle_consistency_loss (Y->X)
iter = 10000 Patch with cycle_consistency_loss (X->Y)	iter = 10000 Patch with cycle_consistency_loss (Y->X)

EXP 4 -- difference between the results with and w/o the cycle consistency loss

We can see that cycle consistency loss plays a critical role in the generation process. Without it, there is a noticeable mismatch between the generated image and its original counterpart. This discrepancy arises because, in its absence, the generator attempts to match with any images in the target domain indiscriminately. Cycle consistency loss effectively bridges this gap by ensuring a meaningful correspondence is maintained.

Taking the "apple2orange" example, without cycle consistency loss, the generated images lack spatial awareness regarding the location of oranges. Consequently, the algorithm tends to indiscriminately alter the outputs, demonstrating the indispensable nature of cycle consistency loss in guiding the transformation process accurately.

Using a cat as an example, the poses of the generated cats failed to align with the baseline cat pose in the absence of cycle consistency loss.

Furthermore, without cycle consistency loss, the background color of the cat images varied unpredictably, often resulting in overly colorful alterations.

EXP 5 -- apple2organge DC test

iter = 10000 grumpifyCat using DC Discriminator (X->Y)	iter = 10000 grumpifyCat using DC Discriminator (Y->X)
iter = 10000 grumpifyCat using Patch Discriminator (X->Y)	iter = 10000 grumpifyCat using Patch Discriminator (Y->X)
iter = 10000 grumpifyCat using DC Discriminator (X->Y)	iter = 10000 grumpifyCat using DC Discriminator (Y->X)
iter = 10000 apple2orange using Patch Discriminator (X->Y)	iter = 10000 apple2orange using Patch Discriminator (Y->X)

A Patch discriminator analyzes the input image by dividing it into smaller sections and evaluating each patch individually to ascertain its authenticity. This method affords the Patch discriminator the capability to offer more granular feedback to the generator, pinpointing specific areas of the image that require enhancement.

Conversely, a DC discriminator evaluates the entire input image as a whole, delivering feedback to the generator based on the global quality of the image.

In the context of our task, as discussed in the preceding section, the DC discriminator outperforms the Patch discriminator. This superiority stems from its ability to maintain the overall structural integrity of the image. For instance, it is adept at recognizing human faces and identifying fruits, highlighting its efficacy in tasks requiring a holistic understanding of the image's composition.