16-825 Assignment 3: Volume Rendering, NeRF, and 3D Gaussian Splatting

Name: Hiroki Mii

Andrew ID: hmii

A. Neural Volume Rendering

1. Differentiable Volume Rendering

1.3 Ray Sampling

xy_grid

xy_grid

rays

rays

1.4 Point Sampling

1.5 Volume rendering

color

depth

2. Optimizing a basic implicit volume

2.2 Loss and training

Box center: (0.25, 0.25, 0.00)

Box side lengths: (2.01, 1.50, 1.50)

2.3 Visualization

3. Optimizing a Neural Radiance Field (NeRF)

nerf

4. NeRF Extras

4.1 View Dependence

View dependence lets a point's color change with viewing angle, which helps with shiny surfaces like the materials scene. However, because color can now vary by direction, the network can overfit to the training data and not learn the 3D geometry correctly, leading to poor-quality renderings from unseen views.

B. 3D Gaussian Splatting

5. 3D Gaussian Rasterization

5.5 Perform Splatting

6. Training 3D Gaussian Representations

6.2 Perform Forward Pass and Compute Loss

part_6_training_progress.gif

part_6_training_final_renders.gif

pre_act_opacities: lr = 0.05

pre_act_scales: lr = 0.005

colours: lr = 0.05

means: lr = 0.0005

num_iter: 1000

PSNR = 27.796, SSIM = 0.923

7. Extensions

7.1 Rendering Using Spherical Harmonics

DC-Only

View 12

View 23

Spherical Harmonics

View 12

View 23

In the View 12 comparison, the spherical harmonics render has darker, more saturated green and yellow, giving better color contrast, while the DC-only render looks lighter and flatter. The DC term only gives each Gaussian its average color over all directions, while the higher-order terms add the view-dependent variation.

In the View 23 comparison, the DC-only render has orange noise on the left side near the seat, while it is more reduced in the spherical harmonics render. Again, this is because DC-only uses each Gaussian's average color, while the higher order terms of the spherical harmonics correct the color for this viewing direction.