16-825 Assignment 4

Guying Lin (guyingl)

1.1 3D Gaussian Rasterization

1.2 Training 3D Gaussian Representations

This is the parameters:

parameters = [
        {'params': [gaussians.pre_act_opacities], 'lr': 0.05, "name": "opacities"},
        {'params': [gaussians.pre_act_scales], 'lr': 0.005, "name": "scales"},
        {'params': [gaussians.colours], 'lr': 0.01, "name": "colours"},
        {'params': [gaussians.means], 'lr': 0.001, "name": "means"},
    ]

It’s trained with 1000 iterations. And these are the metrics:

[*] Evaluation --- Mean PSNR: 30.363
[*] Evaluation --- Mean SSIM: 0.944

1.3 Extension

1.3.1 Rendering Using Spherical Harmonics

The top gif is the one without view-dependent effects. The bottom gif is the one with view-dependent effects.

The side-by-side image comparisons are shown below. The left one is without view-dependent effects. The right one is the with view-dependent effects.

Enabling spherical harmonics makes the chair’s surface pattern in the lower image appear much richer and more realistic — for instance, the color variation and fine decorative textures are more clearly visible compared to the left image. My interpretation is that without modeling view-dependent effects, the rendering tends to smooth out lighting differences across views, which causes subtle patterns to blur together.

Here’s another viewing angle that supports this idea: the chair’s decorative details are noticeably sharper in the left image.

1.3.2 Training On a Harder Scene

For the naive implementation, the result is very blurry. I attach the results of the improved implementation here.

Improved version (1000 iterations)

parameters = [
    {'params': [gaussians.pre_act_opacities], 'lr': 0.05, "name": "opacities"},
    {'params': [gaussians.pre_act_scales], 'lr': 0.005, "name": "scales"},
    {'params': [gaussians.colours], 'lr': 0.01, "name": "colours"},
    {'params': [gaussians.means], 'lr': 0.001, "name": "means"},
    {'params': [gaussians.pre_act_quats], 'lr': 0.001, "name": "quats"},
]

[*] Evaluation --- Mean PSNR: 22.758
[*] Evaluation --- Mean SSIM: 0.816

2.1 SDS Loss + Image Optimization

Prompt: “a hamburger” Left: no guidance | Right: with guidance, 2000 iterations

Prompt: “a standing corgi dog” Left: no guidance | Right: with guidance, 2000 iterations

Prompt: “a cake” Left: no guidance | Right: with guidance, 2000 iterations

Prompt: “a dancing cat” Left: no guidance | Right: with guidance, 2000 iterations

2.2 Texture Map Optimization for Mesh

Left: gray metallic cow | Right: yellow fluffy cow