Basic Transform:
basic_transform = transforms.Compose([
transforms.Resize(opts.image_size, Image.BICUBIC),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
])
Deluxe Transform:
deluxe_transform = transforms.Compose([
transforms.Resize(osize, Image.BICUBIC),
transforms.RandomCrop(opts.image_size),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
])
self.up_conv1 = nn.Conv2d(in_channels=noise_size, out_channels=256, kernel_size=2, stride=1, padding=2)
self.bn1 = nn.InstanceNorm2d(256)
self.up_conv2 = up_conv(in_channels=256, out_channels=128, kernel_size=3, stride=1, padding=1, scale_factor=2, norm="instance", activ="relu")
self.up_conv3 = up_conv(in_channels=128, out_channels=64, kernel_size=3, stride=1, padding=1, scale_factor=2, norm="instance", activ="relu")
self.up_conv4 = up_conv(in_channels=64, out_channels=32, kernel_size=3, stride=1, padding=1, scale_factor=2, norm="instance", activ="relu")
self.up_conv5 = up_conv(in_channels=32, out_channels=3, kernel_size=3, stride=1, padding=1, scale_factor=2, norm=None, activ="tanh")
self.conv1 = conv(3, 32, 4, 2, 1, norm, True, 'relu')
self.conv2 = conv(in_channels=32, out_channels=64, kernel_size=4, stride=2, padding=1, norm=norm, init_zero_weights=False, activ="leaky")
self.conv3 = conv(in_channels=64, out_channels=128, kernel_size=4, stride=2, padding=1, norm=norm, init_zero_weights=False, activ="leaky")
self.conv4 = conv(in_channels=128, out_channels=256, kernel_size=4, stride=2, padding=1, norm=norm, init_zero_weights=False, activ="leaky")
self.conv5 = conv(in_channels=256, out_channels=1, kernel_size=4, stride=2, padding=1, norm=None, init_zero_weights=False, activ=None)
Left - DCGAN + Basic Augmentation | Right: DCGAN + Basic Augmentation + Differential Augmentation
DCGAN & Deluxe Augmentation & Differential Augmentation:
Left: DCGAN + Deluxe Augmentation | Right: DCGAN + Deluxe Augmentation + Differential Augmentation
Utilizing deluxe augmentation and differentiable augmentation significantly improves image quality. Differentiable augmentation, in particular, addresses the Generator's challenges in replicating augmented images, thereby reducing artifacts and enhancing convergence and quality through discriminator regularization. The DCGAN's progress is evident from the decreasing losses of both the discriminator and generator. Notably, the discriminator loss is higher, and the generator loss is lower when differentiable augmentation is used, attributed to the improved regularization. The contrast in loss curves between basic and deluxe augmentation strategies is not substantial.
D_Total:
G_Total:
The initial high G loss curve indicates the Generator's struggle to create convincing images, which gradually decreases as its performance improves. However, unlike the D loss, the G loss may continue to show fluctuations or a downward trend, reflecting the Generator's ongoing efforts to deceive the Discriminator. The use of deluxe data processing and data augmentation results in a lower G loss curve, signifying better image generation. The DCGAN's results after 6,400 iterations, with both basic and deluxe data preprocessing and differential augmentations, show that differential augmentations enhance image quality by reducing artifacts and increasing lifelike features. This improvement is noticeable in later training stages, where images exhibit greater realism and detail, indicating the GAN's progress in generating images that increasingly resemble real ones.
Both Results are from the config: DCGAN + Deluxe Augmentation + Differential Augmentation
Left: Iteration 200 | Right: Iteration 6400
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
The inclusion of cycle-consistency loss in the CycleGAN model significantly improves the realism of the generated images, ensuring they closely resemble the original input in both content and style. This loss function is crucial for maintaining the integrity of the input's domain characteristics, leading to more authentic and visually appealing results. It also aids in preserving essential attributes and textures, resulting in coherent and detailed outputs that narrow the gap between generated and real images. Moreover, cycle consistency loss enhances image quality, evident in sharper details and reduced artifacts, which we can see fromm the above results. It also introduces greater diversity in the results, as seen in the cats dataset, where the model effectively captures facial features.
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X
The DC-Discriminator tends to produce less realistic images compared to the patch discriminator, which assesses images on a patch-by-patch basis. This localized approach enables the patch discriminator to focus on fine-grained details and textures, leading to more nuanced assessments of image realism. In contrast, the DC-Discriminator evaluates the image as a whole, which may result in overlooking finer details that contribute to the perception of realism. Our results have shown that the patch discriminator approach results in sharper images, while the DC-Discriminator approach exhibits a slight decrease in sharpness, with more blurred and artifact-laden images. However, the difference between the two approaches is subtle, as the patch discriminator ensures the authenticity of each image segment, while the DC-Discriminator focuses on the overall authenticity of the entire image, potentially compromising the authenticity of individual subsections.
I used the Stable Diffusionn 2.1 Demo on HuggingFace to generate these outputs:
Left: X2Y | Right: Y2X
Left: X2Y | Right: Y2X