16825 Assignment 4

Andrew ID: nleone

Collaborators: ChatGPT (for GPU memory management)

Note: I only had 8 GB GPU (RTX 3070) for this assignment

Q1

Q 1.1.5

Q 1.2.2

Learning Rates:

gaussians.pre_act_opacities: 1e-3

gaussians.pre_act_scales: 1e-3

gaussians.colours: 5e-3

gaussians.means: 1e-4

Iterations: 1000

Mean PSNR: 27.794

Mean SSIM: 0.925

Q1.3.1

Frame 0 (Left: Default | Right: Harmonic)

Frame 3 (Left: Default | Right: Harmonic)

As seen in Frames 0 and 3, the reflection and shadows of the original rendering do not change based on viewing direction. However with the spherical harmonics rendering, we can see parts of the chair that are darker in Frame 0 are brighter in Frame 3 and vice versa.

In Frame 0, the default rendering has a bright top half and a dark bottom half of the green cushion, while the harmonics rendering has a constant reflection. In Frame 3, the halves have the same reflectance while in the spherical harmonics, the top right corner has the most reflectance, while the rest of the green felt is dark

Q2

Q2.1

All images below were trained with 1000 iterations

a hamburger (with guidance)

a hambuger (no guidance)

a standing corgi dog

peter_family_guy

master shake from aqau teen hunger force

Q2.2

NOTE: get_mesh_renderer_soft: sigma = 1e-4, RasterizationSettings/face_per_pixel = 25

Downsized render image input from 512 x 512 to 256 x 256 (SDS.H = 256, SDS.W = 256)

blue

an orange golden bull

Q2.3

NOTE: Downsized render image input from 512 x 512 to 256 x 256 (SDS.H = 256, SDS.W = 256)

NERF Renderer: staged = False

Parameters: --latent_iter_ratio 0.1 --max_ray_batch 128 --num_steps 16 --h 50 --w 50

a_hamburger

a_standing_corgi_dog

>

Q2.4.3

Before computing the MSE Loss, I decode the latents back into pixel space. However, due to the extra compute needed to decode the latents, I had to reduce the NERF render image heights to 32 x 32 and set the precision of the SDS model to float16. Due to the lack of RAM, I was unable to successfully train with this loss.

a_standing_corgi_dog

>