Assignment 3: Volume Rendering, NeRF, and Gaussian Splatting

Shruti Jain · spjain

1. Differentiable volume rendering

1.3. Ray sampling

NDC pixel grid
Absolute ray directions as RGB

1.4. Point sampling

Sampled points from the first camera

1.5. Volume rendering

Volume-rendered box orbit
Depth normalized by its maximum

2. Optimizing an implicit volume

2.2. Loss and training

Center: (0.25, 0.25, 0.00). Side lengths: (2.01, 1.50, 1.50). Final sampled RGB MSE: 9.63e-7.

2.3. Optimized volume

Optimized box orbit

3. Neural radiance field

Learned Lego scene

4.1. View dependence

Without view dependence With view dependence
Materials scene without view dependence Materials scene with view dependence

Allowing color to depend on viewing direction helps represent highlights and reflections. A more expressive directional branch can fit training images more closely, but may memorize view-specific appearance and generalize poorly to unseen cameras. I keep density dependent only on position and limit the direction input to a small color branch with two harmonic frequencies. This preserves shared geometry while limiting appearance flexibility, at the cost of potentially blurring sharp reflections.

4.2. Coarse/fine sampling (extra credit)

Compared with uniform sampling, coarse/fine 64 + 64 improves PSNR from 24.75 to 25.03 dB, but increases rendering time from 278.1 to 424.2 ms per image (about 53% slower). Coarse/fine 32 + 32 reduces rendering time to 214.6 ms per image (about 23% faster than uniform sampling), but PSNR drops to 23.94 dB.

Sampling Network queries per ray PSNR Render time per image
Uniform control 128 24.75 dB 278.1 ms
Coarse/fine: 64 + 64 192 25.03 dB 424.2 ms
Coarse/fine: 32 + 32 96 23.94 dB 214.6 ms
Uniform control Coarse/fine: 64 + 64 Coarse/fine: 32 + 32
Uniform control Coarse/fine 64 + 64 Coarse/fine 32 + 32

5. Gaussian rasterization

5.5. Splatting

DC-only Gaussian chair rendering

6. Training Gaussian representations

6.2. Training and evaluation

Parameter Learning rate
means 0.00016
scales 0.005
colours 0.0025
opacities 0.05

Iterations: 1,000

PSNR: 29.349 dB · SSIM: 0.9388

Training progress

Truck training progress

Final renders

Final truck views

7.1. Spherical harmonics

Without spherical harmonics (DC-only) With spherical harmonics
Gaussian chair without spherical harmonics (DC-only), from Q5.5 Gaussian chair with spherical harmonics

Frame 0 — Left: DC-only · Right: Full SH

Same-view comparison, frame 0

Frame 0: The chair with full spherical harmonics has stronger contrast between the green and gold colors, and the shading transitions appear smoother. The gold pattern is more distinct, so the upholstery details appear better preserved.

Frame 8 — Left: DC-only · Right: Full SH

Same-view comparison, frame 8

Frame 8: With full spherical harmonics, the green upholstery looks darker and the gold pattern stands out more clearly. The stronger contrast makes the fabric details easier to see, although streaks around the thin supports remain.

Frame 20 — Left: DC-only · Right: Full SH

Same-view comparison, frame 20

Frame 20: The green and gold trim has stronger contrast with full spherical harmonics, making its pattern easier to see. The beige back panel looks similar in both images, and the streaks around the thin supports remain.