16-825 Assignment 3: Volume Rendering, NeRF, and 3D Gaussian Splatting

Andrew ID: zizheng2

1. Differentiable Volume Rendering

256 × 256 images, 64 samples per ray. The depth map is normalized by its maximum value.

Pixel coordinate grid
Pixel grid
World-space ray directions
Rays
Points sampled along rays
Sample points
Normalized depth map
Normalized depth
Rendered implicit box
Box rendering

2. Optimizing a Basic Implicit Volume

1000 epochs, Adam learning rate 0.0005, RGB MSE. Final parameters, rounded to two decimal places:

Parameterxyz
Center0.250.250.00
Side lengths2.011.501.50
Optimized box orbit
Optimized box

The recovered box has the expected longer x dimension and displaced center, consistent with the reference rendering.

3. Neural Radiance Field

Lego, 128 × 128, 250 epochs, 1024 rays per step, 128 samples per ray, initial learning rate 0.0005. The position-only MLP uses positional encoding and RGB MSE.

Lego NeRF orbit
Lego NeRF

4.1 View Dependence

Materials, 128 × 128, 250 epochs, initial learning rate 0.0001. Both runs use 1024 rays per step and 128 samples per ray. Viewing direction is added only to the color branch; density depends on position.

Materials without viewing direction
Without view dependence
Materials with viewing direction
With view dependence

Direction input changes brightness and highlight contrast on some spheres. It can represent reflections that a single color per point cannot, but extra directional capacity can overfit training views. Keeping density independent of direction and using a small color branch limits this risk. The visual difference is modest at this resolution.

4.3 VolSDF

Lego, 128 × 128, 250 epochs, initial learning rate 0.0005, 128 samples per ray, α = 10, Eikonal weight = 0.02. I compare three β values with the other settings fixed.

βObservation
0.01Large floating structures and background clutter.
0.05 — selectedBest visual balance of color and shape; background fragments remain.
0.10Softer color and incomplete geometry.
Selected VolSDF color result at beta 0.05
Color, β = 0.05
Selected VolSDF geometry at beta 0.05
Geometry, β = 0.05

α controls interior density and opacity. β controls the width of the density transition around the zero-level surface.

  1. High vs. low β: High β favors a broad, diffuse boundary; low β concentrates density changes near the surface and favors a sharper boundary.
  2. Which is easier to train? High β gives gradients over a wider region. Low β has a narrow transition that fixed ray samples can miss.
  3. Which gives a more accurate surface? Low β can localize the surface better when sampling and optimization are adequate. Here, 0.01 produced more artifacts, so I select 0.05. The selected mesh still contains unwanted geometry.

5. 3D Gaussian Rasterization

The four provided unit tests pass. Pretrained chair, 256 × 256, 2048 Gaussians per splat. Each frame shows RGB, depth, and silhouette from left to right.

Chair RGB, depth, and silhouette without spherical harmonics
DC color only

6. Training 3D Gaussian Representations

Truck, isotropic Gaussians initialized from the supplied point cloud, 1000 iterations, Adam, L1 RGB loss.

Mean LRLog-scale LROpacity-logit LRColor LR
0.00010.0050.050.01
Test PSNRTest SSIM
29.438 dB0.938
Truck training progression with predictions above ground truth
Training progress: predictions on top, ground truth below.
Final truck renderings
Final renderings

7.1 Rendering Using Spherical Harmonics

I render the pretrained degree-3 SH coefficients. The DC-only GIF is shown in Part 5.

Chair RGB, depth, and silhouette with spherical harmonics
With spherical harmonics

Same camera in each pair. Left: DC only. Right: full SH.

Frame 0: DC only left, full SH right
Frame 0. SH darkens the green upholstery and strengthens the gold pattern on the backrest.
Frame 10: DC only left, full SH right
Frame 10. The seat is darker and its pattern has more contrast with SH. Edge streaks remain in both.
Frame 21: DC only left, full SH right
Frame 21. The beige underside changes little; the difference is smaller than on the front upholstery.