Project3 When Cats meet GANs

 

Lena Du

 

Part 1 DCGAN

 

1.1 Implement Data Augmentation

Deluxe data augmentation helps the model to be more robust.

 

1.2 DCGAN - Discriminator

Padding

The calculation of padding is:

m=n+2pKS

, where m is the output size and n is the input size. p is the padding, K is the kernel size, and S is the stride. Given the size is downsampled by scale 2, we know n = 2m. With K = 4 and S = 2, we will have

2m=2m+2p4

 

Then,

p=1

which means padding is 1.

 

1.3 DCGAN - Generator

The design of the first layer in DCGenerator is using conv, instead of up_conv. The idea is to use padding 3, kernel size 4, and stride 1 to obtain a 4x4 output. I also replaced nn.ReLU with nn.LeakyReLU for its better performance.

 

1.4 Result

As we can see, the result of the Deluxe data augmentation + full diffaug configuration with more iterations has better quality and resolution.

ConfigReal Image1000 Iterations7000 Iterations
Basicbasic_realbasic_realbasic_real
Deluxebasic_realbasic_realbasic_real
Deluxe + diffaug (cutout)basic_realbasic_realbasic_real
Deluxe + diffaug (color, translation)basic_realbasic_realbasic_real
Deluxe + diffaug (color, cutout)basic_realbasic_realbasic_real
Deluxe + diffaug (color, translation, cutout)basic_realbasic_realbasic_real

 

ConfigLoss
Basicbasic
Deluxebasic
Deluxe + diffaug (all)basic

 

 

Part 2 CycleGAN

Observations:

 

2.1 Cat

2.1.1 DC Discriminator /wo Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

2.1.2 DC Discriminator /w Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

2.1.3 Patch Discriminator /wo Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

 

2.1.4 Patch Discriminator /w Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

2.2 Fruits

2.2.1 DC Discriminator /wo Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

2.2.2 DC Discriminator /w Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

2.2.3 Patch Discriminator /wo Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

 

2.2.4 Patch Discriminator /w Cycle Consistency
Direction1000 Iterations5000 Iterations10000 Iterations
X -> Ysample-001000-X-Ysample-001000-X-Ysample-001000-X-Y
Y -> Xsample-001000-X-Ysample-001000-X-Ysample-001000-X-Y

 

2.3 Loss

DiscriminatorSmoothness 0.972
DC /wo Cycle Consistencydc_cyclegan_NOconsist
DC /w Cycle Consistencydc_cyclegan_NOconsist
Patch /wo Cycle Consistencydc_cyclegan_NOconsist
Patch /w Cycle Consistencydc_cyclegan_NOconsist

 

DiscriminatorSmoothness 0.999
DC /wo Cycle Consistencydc_cyclegan_NOconsist
DC /w Cycle Consistencydc_cyclegan_NOconsist
Patch /wo Cycle Consistencydc_cyclegan_NOconsist
Patch /w Cycle Consistencydc_cyclegan_NOconsist

Part 3 B&W

Spectral loss

ConfigReal Image1000 Iterations7000 Iterations
Deluxe + Instancebasic_realbasic_realbasic_real
Deluxe + Spectralreal-001000real-001000real-001000