Assignment 3: Volume Rendering, NeRF, and 3D Gaussian Splatting

1.3 ray sampling

1.4 point sampling

1.5 Volume rendering

2.3 Optimizing a basic implicit volume

- Center: (0.25, 0.25, -0.00) - Side lengths: (2.01, 1.50, 1.50)

The gif is, more or less, the same as the TA solution.

3: Optimizing a Neural Radiance Field (NeRF)

4.1 View Dependence

Adding view dependence makes the renders more realistic, since surfaces that should look brighter or darker from different angles (like the highlights on the shiny spheres) now actually do. However, in a similar manner to what happens when we make the network larger, we run the risk of overfitting, as the network could explain each training image with view-specific colour instead of learning one consistent scene. As we heard in class, the result is that renders from the training views look good, but new views carry a greater risk of looking wrong. To mititigate this, we add the viewing direction AFTER the density is computed (so the geometry is the same from every angle). therefore it only gets a few harmonic frequencies, so colour can only change smoothly with angle.

5 3D Gaussian Rasterization

6 Training 3D Gaussian Representations

opacities 4e-2, scales: 4e-3, colors: 4e-3, means: 4e-4

training iterations of 1000

psnr = 29.962 and ssim = 0.943

7.1 Spherical harmonics

Left: view-independent rendering from Q5.5 (DC spherical-harmonic term only). Right: rendering with the full spherical-harmonic color (Q7.1), both produced with render.py on the same pretrained scene.

Side-by-side RGB comparisons for the same frames. In each row: Q5.5 view-independent render (left), Q7.1 spherical-harmonics render (middle), and the per-pixel absolute difference scaled by 4 (right).

Frame 0 (front view)

The view-independent render is flatter and more "yellow green", since each Gaussian is drawn with one color no matter where the camera is. With SH the color depends on the viewing direction, so the fabric comes out a deeper green and the gold pattern shows more. The difference image covers all but the wooden frame (as it is almost black), so the wood looks about the same from all other angles, thus, a singluar color is enough for it..

Frame 8 (looking down from above/behind)

From this steep angle the top ehdge of the backrest appears as a very bright, almost neon green strip in the view-independent render, and the seat cushion is lighter than it should be. The extra credit rendes includes a darker cushion with a goldish yellowish pattern on it. The difference is concentrated on that strip and on the seat; the feathery floater Gaussians around the silhouette are the same in both, which is expected since the geometry and opacities are unchanged and only the color evaluation differs.