Part 1: Inverting the Generator [30 pts]¶

  1. Varying the L1 weight terms

| 1 | 0.1 | 0 | 1000 | 100 | --- | --- | --- | --- | --- | Alt Text | Alt Text | Alt Text | Alt Text | Alt Text |

From above L1 weight as 0.01 gives the best result.

  1. Varying the Perceptual loss weight

| 9 | 1 | 0.1 | 0.01 | 0.001 | --- | --- | --- | --- | --- | Alt Text | Alt Text | Alt Text | Alt Text | Alt Text |

From above Perceptual loss weight as 10 gives the best result.

  1. Varying the Architecture

| VanillaGAN | StyleGAN | --- | --- | | Alt Text | Alt Text

StyleGAN gives better results.

  1. Varying the Latent Space

| z | w | w+ | --- | --- | --- | | Alt Text | Alt Text | Alt Text

w+ captures better color details.

Inference Time: All models take approximately 18-20 seconds to run.

Part 2: Scribble to Image [40 Points]¶

| Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text | --- | --- | --- | | Alt Text | Alt Text | Alt Text

Findings from above:

  • Sparse masks works better compared to dense masks. With dense mask more number of pixels are enforced to match the sketch, and since the sketch is unrealisitc makes it harder for model to converge.

Part 3:¶

Results:

| 1 | 2
| --- | --- | | Alt Text | Alt Text

  1. Varying the Noise

| Sketch | 100 | 250 | 500 | --- | --- | --- | --- | Alt Text | Alt Text | Alt Text | Alt Text |

More noise helps make the generated image farther away from the sketch.

  1. Varying the strenght of classifier-free guidance

| Sketch | 10 | 15 | 30 | 60 | --- | --- | --- | --- | --- | Alt Text | Alt Text | Alt Text | Alt Text | Alt Text |

Increasing the strength of classifier-guidance makes the cat more grumpy i.e. more aligned to the prompt.

Bells & Whistles (Extra Points)¶

| BW1 | BW2 | --- | --- | Alt Text | Alt Text |

In [ ]: