Assignment 3

lnayak

1. Differentiable volume rendering

1.3 Pixel grid
1.3 Pixel grid
1.3 Camera rays
1.3 Camera rays
1.4 Samples along rays
1.4 Samples along rays
1.5 Volume rendering
1.5 Volume rendering
1.5 Normalized depth
1.5 Normalized depth

2. Optimizing an implicit volume

Box center: (0.25, 0.25, 0.00). Side lengths: (2.01, 1.50, 1.50).

Optimized box
Optimized box

3. Neural radiance field

Lego

Lego
Lego

4.1 View dependence

View dependence lets highlights change with the camera direction, but can also overfit the training views. The table compares performance on held-out views.

RunEpochsStepsTest PSNR (dB)Test SSIMStep (s)Render/view (s)
Position only2502500022.9500.84110.01020.0811
Position and direction2502500023.5020.85330.01060.0863

Materials: position only

Materials: position only
Materials: position only

Materials: position and direction

Materials: position and direction
Materials: position and direction

Second run minus first run: test PSNR (dB): +0.552; test SSIM: +0.0123; seconds/step: +0.0004; seconds/rendered view: +0.0051.

4.2 Coarse and fine sampling

The coarse network guides extra samples toward occupied regions. The fine network uses these samples to recover detail, at the cost of another network pass.

RunEpochsStepsTest PSNR (dB)Test SSIMStep (s)Render/view (s)
Uniform (Q3)2502500024.2420.88660.01010.0791
Coarse/fine2502500024.6620.89450.01880.1522

Uniform: 128 samples per ray. Coarse/fine: 64 coarse and 128 fine samples per ray.

Coarse/fine Lego rendering
Coarse/fine Lego rendering

Second run minus first run: test PSNR (dB): +0.420; test SSIM: +0.0078; seconds/step: +0.0087; seconds/rendered view: +0.0731.

4.3 VolSDF

Alpha controls density: a larger value makes the volume more opaque. Beta controls the transition around the surface. High beta produces a thicker, softer boundary; low beta produces a sharper boundary.

High beta is usually easier to train because gradients reach samples farther from the surface. Low beta can give a more accurate surface, but needs sufficiently dense samples near it.

Lego VolSDF

Lego VolSDF
Lego VolSDF
VolSDF surface geometry
VolSDF surface geometry

This run uses alpha=10.000, beta=0.020 and Eikonal weight=0.050. Learning rate: 0.0005; 500 epochs, 1024 rays per step, 128 samples per ray. The smooth density transition helps optimization, while the Eikonal loss encourages unit-length SDF gradients.

InitializationBetaEikonal weightEpochsVal PSNR (dB)Val SSIM
Random0.050.0225021.2240.7704
Geometric0.050.0225021.0330.7690
Geometric0.020.0550021.4940.7965
Random0.020.0525021.2850.7796
Random0.10.0225020.7350.7339

I initialized the distance network to approximate a sphere, then trained all its weights. This avoided the distant surfaces that appeared with random initialization. I selected the result using validation scores and the rendered surface.

5. Gaussian rasterization

DC colours: RGB, depth and silhouette
DC colours: RGB, depth and silhouette

6. Training Gaussian representations

Truck

RunIterationsGaussiansCovarianceSplitViewsPSNR (dB)SSIM
Truck100010000isotropictest529.2750.9388
ParameterInitial learning rateFinal learning rate
opacities0.050.05
scales0.0050.005
colours0.00250.0025
means0.000160.00016

Loss: L1. Learning-rate schedule: constant.

Truck: training progress, prediction above and ground truth below.
Truck: training progress, prediction above and ground truth below.
Truck: final renderings.
Truck: final renderings.

7.1 Spherical harmonics

SH colours: RGB, depth and silhouette
SH colours: RGB, depth and silhouette
Frame 000. DC on the left, SH on the right. With SH, the backrest has darker green areas and a clearer gold pattern. The white frame changes little.
Frame 000. DC on the left, SH on the right. With SH, the backrest has darker green areas and a clearer gold pattern. The white frame changes little.
Frame 008. DC on the left, SH on the right. SH increases the contrast of the gold pattern on the backrest and seat edge. The silhouette stays the same.
Frame 008. DC on the left, SH on the right. SH increases the contrast of the gold pattern on the backrest and seat edge. The silhouette stays the same.
Frame 016. DC on the left, SH on the right. The backrest has stronger light and dark variation with SH, while the seat appears slightly brighter.
Frame 016. DC on the left, SH on the right. The backrest has stronger light and dark variation with SH, while the seat appears slightly brighter.

7.2 Harder scene

Materials baseline

RunIterationsGaussiansCovarianceSplitViewsPSNR (dB)SSIM
Materials baseline100010000isotropicvalidation10017.7640.6505
ParameterInitial learning rateFinal learning rate
opacities0.050.05
scales0.0050.005
colours0.00250.0025
means0.000160.00016

Loss: L1. Learning-rate schedule: constant.

Materials baseline: training progress, prediction above and ground truth below.
Materials baseline: training progress, prediction above and ground truth below.
Materials baseline: final renderings.
Materials baseline: final renderings.

Materials modified

RunIterationsGaussiansCovarianceSplitViewsPSNR (dB)SSIM
Materials modified300010000anisotropicvalidation10023.6020.8635
ParameterInitial learning rateFinal learning rate
opacities0.050.05
scales0.0050.0005
colours0.010.01
means0.0030.0003
quats0.0010.001

Loss: L1. Learning-rate schedule: exponential: means and scales decay to 0.1x by final update.

Materials modified: training progress, prediction above and ground truth below.
Materials modified: training progress, prediction above and ground truth below.
Materials modified: final renderings.
Materials modified: final renderings.
SettingBaselineModified
IsotropicTrueFalse
Iterations10003000
Learning-rate scheduleconstantexponential: means and scales decay to 0.1x by final update
Initial position standard deviation0.20000.7000
Initial opacity0.99970.1000

Modified minus baseline: PSNR +5.838 dB; SSIM +0.2131. The modified run has the higher PSNR. Anisotropic Gaussians can stretch along different directions. Learning-rate decay makes the final position and scale updates smaller. Larger position updates help points reach the objects. A higher colour learning rate helps their appearance converge. The modified run also trains for more iterations. I spread the initial random points more widely to cover the scene. Lower initial opacity lets points behind the foreground points contribute during early training.

The modified run separates the objects much more clearly than the baseline. Some reflections remain softer than the ground truth; each Gaussian still has one RGB colour for all viewing directions.