Assignment #5 - Cats Photo Editing

Rohan Choudhury

I. Inverting the Generator

Combinations of Loss Types

Effect of different loss types on reconstruction.

The first row is with L1 loss only, the second row is with a combo of L1 and perceptual loss, with 0.05 weight on perceptual, the third row is with both, but with a much higher weight on L2, and the fourth row is perceptual only. With only perceptual loss, there are a lot more artifacts, but with L2 loss, the image is a bit blurrier. It seems that a combination with lower weight on the perceptual loss performs best.

Effect of Generative Model Type

Samples with different model types.

The top row is the image generated with StyleGan, the second row is Vanilla GAN, and the final row is the base image. We see that generally StyleGan performs the best.

Effect of Latent Space Code

Samples with differnet latent code types.

The top row is with z, the second row with w, and the third with w+. Each row has images that it looks the best with, but generally w+ performs best. the z code introduces some large artifacts, and w is not as good at following the guidance.

II. Scribble To Image

Samples of sketches and their output

Sample sketches to drawings.

Generally the generated images follow the guidance sketches. However, the less color, the worse the generated images look, and the more saturated the images look.

III. Stable Diffusion

Diffusion samples

Style transfer results with all the input content and style images.

Above are samples on input sketches. First, we have a grumpy cat, reimagined as a royal painting. Second, we have an anime style sketch of a girl, reimagined in the style of Studio Ghibli. In both cases the similarites between the input drawing and the output are very visible.

Effect of noise

Optimizing random input content vs a style.

The noise has a huge effect. For 500, the image is essentially the samea s the guide. For 700, you can start to see a difference. Finally, for 1000 steps, the prompt dominates, and you can still see the effect of the guide but the output is much more like what is desired.

Effect of guidance strength

Some examples on my own images.

The effect of the guidance strength seems to be quite large. The more guidance is provided, the more influence is visible from the original image. All of the above images were generated with 999 steps of noise.