Assignment 3: Volume Rendering, NeRF, and 3D Gaussian Splatting
Andrew ID: djonnala
A. Neural Volume Rendering
1. Differentiable Volume Rendering
1.3 Ray Sampling
xy_grid
rays
1.4 Point Sampling
1.5 Volume Rendering
Color
Depth
The image appears to be split horizontally, with a small phase difference between the two halves. This comes from the chunk size in the config. Setting the chunk size to W*H matched the TA output.
2. Optimizing a basic implicit volume
2.2 Loss and Training
x
y
z
Box Center
0.25
0.25
0.00
Box Side Length
2.01
1.51
1.52
2.3 Visualization
The visual matches the reference, except for the horizontal split and phase difference from chunking noted in 1.5.
3. Optimizing a Neural Radiance Field
I trained for about 50 epochs and got a result that matches the reference image. I used the network from the NeRF paper as a reference and implemented it from scratch. I first forgot the sigmoid on color, so the render stayed black. After adding it, the network trained well.
4.1 View Dependence
As in the original NeRF paper, the model takes the direction vector d and concatenates it with the output of the first 8 linear layers to predict color.
Initialization mattered here. An early high-resolution run produced a very soft bulldozer after 10 epochs, but it did not checkpoint. When training started again, the model produced a black image the whole time because the density stayed at 0. Kaiming initialization, plus a bias of 0.1 on the density head only, made training much more stable.
NeRF on high-resolution images with view dependence, 450 epochs.
Without View Dependence
With View Dependence
The view-dependent render reacts to the viewing direction and reflects light toward the camera. The model without view dependence does not change color with orientation.
B. 3D Gaussian Splatting
5.5 Perform Splatting
Some high-frequency noise appears only at certain angles.
6.2 Forward Pass and Loss
Number of iterations: 1000. Mean PSNR: 27.157. Mean SSIM: 0.910.
Hyperparameters
Value
iterations
1000
means lr
0.0001
opacity lr
0.005
scales lr
0.005
colors lr
0.001
quats
not trained (isotropic)
Training progressFinal renders
7.2 Training on a Harder Scene
For the best result, I used:
A larger sphere for the random Gaussian initialization.
A high learning rate for means, decayed from 0.01 to 0.0001.
A delayed warmup scheduler on the other parameters, so a Gaussian can latch onto a surface before fitting to it.
A larger learning rate on the means encourages Gaussians to move away from the initial positions.
Iterations
PSNR
SSIM
Final Renders
1000
17.388
0.600
Initializing in a bigger sphere spreads the Gaussians out. The larger learning rate on the means meant they never settled, and they stayed jittery even when they were roughly in the right place.