Original data
Result
Perceptual loss weight
L1 loss weight
L2 loss weight
0.001
10
0
0.01
10
0
0.1
10
0
0.001
0
10
0.01
0
10
0.1
0
10
Here, my loss is a weighted sum of L1, L2, and perceptual losses. All results are produced after 1000 iterations, with stylegan in w+ space. I tune the three weights for a best configuration. The parameters experimented are as follows:
Original data
Result
Perceptual loss weight
L1 loss weight
L2 loss weight
0.001
10
0
0.01
10
0
0.1
10
0
0.001
0
10
0.01
0
10
0.1
0
10
To conclude, the best combination of weights is l1_wgt = 0, l2_wgt = 10, perceptual weight = 0.001.
Using the best hyperparametes of vanilla and stylegan, the results are as follows.
style GAN demonstrates a more accurate reconstruction than vanilla gan.
Data
vanilla
stylegan
Using z, w, and w+ with stylegan model and best hyperparameters.
data
z
w
w+
w+ and w results are better than z results. Especially the w+ result: the green color in the background is most accurately reconstructed. Also, the cat has sharper features and more realistic details in its eyes and its mouth.
sketch
mask
result
A sparser mask produced less sketch-like results and more realistic looking results, because the optimizer alter the pixels inside the mask.
More details in the sketch also contribute to more realistic details, especially the cats' eyes.
With my implementation, more contrast in the sketch produces better results with sharper features.
Here I am using guidance weight = 25, iterations = 750
"Grumpy cat reimagined as a royal painting"
"Mountain scene that looks like a Monet painting"
"Mountain scene that looks like a Monet painting"
noise std / 2
noise std = 1
noise std * 2
With less noise, the result looks more detailed, and the more it is able to reflect the prompt. With an increased amount of noise added to the input, we might need to increase the number of iterations to achieve better results.
iter = 500
iter = 750
iter = 1000
"Grumpy cat reimagined as a royal painting"
strength = 15
strength = 25
strength = 45
The higher the guidance weight, the more the image resembles what is said in the prompt instead of being similar to the original sketch. However, too much guidance weight can lead to the image looking fake.
Here interpolation is done for the 0th and 1st data, the 2nd and 3rd data in the grumpy cat dataset.
