Assignment 3¶
Part 1¶
In discriminator, we apply convolutional layers of stride S=2 and kernel size K=4. We aim to half the spatial dimension of the feature tensor after each convolution. This leads to the following equation:
$ W / 2 = (W + P + S - K) / S $
where W = feature width, P=Padding, S=stride, K=kernel size. For discriminator, we get padding=2 (~ padding=1 on each side and thus P=1 in code). During upsampling, we use K=3, and padding=1 on each side.
Gray: Basic Yellow: Basic + Diff Aug Purple: Deluxe Cyan: Deluxe + Diff Aug
We observe that models that are trained with deluxe or diffAugment lead to higher discriminator losses and lower generator losses. This suggests that these models better fool the discriminator. Qualitatively, in our case this means more realistic fakes. In terms of implementation, differentiable augmentations are applied to both the real and fake images and are differentiable, while the basic augmentations are only applied on the real images and do not need to be differentiable.
Outputs¶
Iteration 200¶
Iteration 6000¶
The initial samples are very noisy while the later samples look much more realistic. Although the samples do look very similar to each other, which could be an indication of mode collapse
Part 2¶
Left: X->Y Right: Y->X
CycleGan without cycle consistency at 1000 iterations¶
CycleGan with cycle consistency at 1000 iterations¶
CycleGan without cycle consistency at 10000 iterations¶
CycleGan with cycle consistency at 10000 iterations¶
CycleGan with cycle consistency at 10000 iterations with DCDiscriminator¶
Apple2Orange¶
CycleGan without cycle consistency at 10000 iterations¶
CycleGan with cycle consistency at 10000 iterations¶
CycleGan with cycle consistency at 10000 iterations with DCDiscriminator¶
As we can see, Cycle consistency helps to perform more local edits, because the objective is to reverse these changes and give back the original images. This is especially important on the orange-apple dataset, where the scenes have more background. Models trained with cycle consistency loss makes less edits on the background and more on the actual objects of interest. additionally, the results are worse when we use the DC discriminator as compared to the patch one. We hypothesize this is because the patch discriminator rewards/penalizes local features, instead of scoring a holistic scene representation.
