Assignment 3
Jianing Zhang (jianing7)
A. Neural Volume Rendering
1.3 Ray sampling (5 points)


1.4 Point sampling (5 points)

1.5 Volume rendering (20 points)


2.2 Loss and training (5 points)
Box center: (0.25, 0.25, 0.00)
Box side lengths: (2.01, 1.50, 1.50)
2.3 Visualization


3 Optimizing a Neural Radiance Field (NeRF) (20 points)

Trained for 250 epochs on 128x128 lego images with the nerf_lego config (6 harmonic functions for position, 6 layers of 128 with a skip connection at layer 3). Training took about 14 minutes and the final loss was about 0.003, roughly 25 dB PSNR. The spiral matches the reference: the shape, the bucket and the tracks are clear, with some blur on fine details at this resolution.
4.1 View dependence (10 bonus points)

nerf_materials_view)
nerf_materials_view_highres)Density depends on position only, so the geometry is the same from every view. The color takes the position feature together with the viewing direction: the feature goes through a linear layer, is concatenated with a harmonic embedding of the direction (2 frequencies, 15 dimensions), and a small branch (64 hidden units, ReLU, then sigmoid) outputs the color. I trained on the materials scene for 250 epochs, first at 128x128 (nerf_materials_view). At 128x128 each sphere is only about 10-15 pixels wide and no highlight is visible. Training at 400x400 (nerf_materials_view_highres, same model, with chunk_size lowered to 4096 to fit in memory), the white highlights on the shiny spheres (silver, copper, teal) get brighter or dimmer depending on the view, while the red and dark spheres stay matte. However, the highlights do not move across the sphere surfaces as the camera moves, as a real specular reflection would. With only 2 direction frequencies, the color can only change slowly and smoothly with the view, so the model learns an overall brightness change per view instead of a sharp highlight that slides over the surface.
More view dependence lets the model fit shiny materials and the training images better, but it also gives the network a way to cheat: it can explain wrong geometry with colors that change per view, which fits the training views and looks wrong from new views. With few training views this overfitting gets worse. To keep it in check, the direction enters late, only for color and not density, and with few frequencies, so the color can only change smoothly with the view.
B. 3D Gaussian Splatting
5 3D Gaussian rasterization (35 points)

6 Training 3D Gaussian representations (15 points)
| parameter | learning rate |
|---|---|
| opacities | 0.05 |
| scales | 0.005 |
| colours | 0.01 |
| means | 0.0005 |
Iterations: 1000
PSNR: 29.357, SSIM: 0.940


7.1 Rendering using spherical harmonics (10 points)




With spherical harmonics the green fabric is darker and more saturated, and the gold brocade pattern on the back stands out more against it. Without them the fabric is a lighter, flatter green and the pattern is washed out.


Same difference: the back and seat are darker and the gold pattern down the middle of the back is much clearer with spherical harmonics. The version without them looks evenly lit, since each Gaussian has one color no matter where it is seen from.


The largest difference of the three. The seat cushion is clearly darker with spherical harmonics and shows its gold pattern, which is barely visible without them. The colors now depend on the viewing direction, so the fabric's sheen and shading change with the view, while the version without spherical harmonics uses the average color from all views.