Andrew ID: yunxianm
I converted the pixel grid to camera NDC coordinates and unprojected it to get world-space rays. The box uses 64 samples per ray between 0 and 5. For each sample, I computed alpha = 1 - exp(-sigma * delta) and multiplied it by the transmittance from earlier samples. Color and depth use these weights.
| Pixel grid | Ray directions |
|---|---|
![]() |
![]() |
| Sample points from the first camera | Depth divided by its maximum |
|---|---|
![]() |
![]() |

I used RGB MSE and Adam with a learning rate of 0.0005. Each update sampled 4096 rays. After 1000 epochs, the box parameters were:
| Parameter | x | y | z |
|---|---|---|---|
| Center | 0.25 | 0.25 | 0.00 |
| Side lengths | 2.01 | 1.50 | 1.50 |

The optimized box looks close to the TA reference, including its proportions and the color pattern around the sides.
The position MLP has six layers with 128 hidden units. It uses six frequency bands for position encoding and concatenates the input encoding before the fourth layer. Density uses ReLU and color uses sigmoid. I initialized the density output bias to 0.1 to avoid starting with an all-zero density field.
I trained on 100 Lego images for 250 epochs at 128x128, with 1024 rays and 128 samples per ray. The starting learning rate was 0.0005, using the provided decay schedule. Evaluation on the 200 test images gave PSNR 24.43 dB and SSIM 0.8872.

For materials, I added a color branch with 64 hidden units and two frequency bands for viewing direction. The density branch still uses only position. Both models used 250 epochs, 128x128 images, and a starting learning rate of 0.0001.
| Model | Train MSE | Test MSE | Test PSNR (dB) | Test SSIM |
|---|---|---|---|---|
| Position only | 0.00470 | 0.00509 | 23.03 | 0.8469 |
| With direction | 0.00400 | 0.00452 | 23.57 | 0.8558 |
The train MSE was evaluated on the 100 training images after training. It fell by 14.9% with the direction branch. Test MSE fell by 11.2%, and test PSNR increased by 0.54 dB. The improvement on new views shows that the branch is useful, but its larger gain on training images suggests that some of the extra fitting is specific to those views. I expect a larger direction branch could fit more detailed reflections, but I did not test that here.

| Position only | With direction |
|---|---|
![]() |
![]() |
I projected the 3D covariance to the image plane and composited the visible Gaussians in depth order. The supplied Gaussian tests passed 4/4.
I changed the provided render() to add only the uncovered background, since the RGB sum is already alpha-weighted. I also include the last alpha when passing transmittance to the next chunk.
The animation shows RGB, depth, and silhouette from left to right.

I trained 10000 isotropic Gaussians from the supplied Truck point cloud for 1000 iterations, using L1 loss. The position learning rate is small because the point cloud already provides an estimate of the geometry.
| Parameter | Learning rate |
|---|---|
| Position | 0.0001 |
| Log scale | 0.005 |
| Color | 0.01 |
| Opacity logit | 0.01 |
On the five test images, mean PSNR was 28.518 dB and mean SSIM was 0.931. The image size was 160x90.
The progress animation has the prediction above the target. The final animation goes through all 197 training views.



Truck test view: target on the left, prediction on the right.
I used degree-3 spherical harmonics, with 16 basis functions for each color channel. The viewing direction points from the camera center to the Gaussian mean. The C0, C1, C2, and C3 constants and basis ordering follow the reference linked in the assignment.


At frame 4, SH makes the green seat darker and the pale flower pattern easier to see. The upper-right part of the backrest is also brighter.

At frame 12, the seat has stronger contrast between the dark green fabric and the light pattern. The backrest also changes in brightness across its surface, while the chair outline stays in the same place.

At frame 20, the large pale underside looks almost unchanged. The small exposed areas of green fabric show slight brightness differences, so the SH effect is much less visible from this angle.
Environment: Python 3.11, PyTorch 2.4.1 with CUDA, and PyTorch3D 0.7.9.
Put materials.png and materials.pth from the course archive in nerf/data/. The Lego loader downloads its data automatically.
Run from the code directory:
cd nerf
mkdir -p images ../results/lego_baseline ../results/materials_baseline ../results/materials_view
python volume_rendering_main.py --config-name=box
python volume_rendering_main.py --config-name=train_box
python volume_rendering_main.py --config-name=nerf_lego training.resume=False training.checkpoint_path=checkpoints/lego_baseline.pth training.checkpoint_interval=249 training.render_interval=249 renderer.chunk_size=4096
cp images/part_3.gif ../results/lego_baseline/orbit.gif
python volume_rendering_main.py --config-name=nerf_materials training.resume=False training.checkpoint_path=checkpoints/materials_baseline.pth training.checkpoint_interval=249 training.render_interval=249 renderer.chunk_size=4096
cp images/part_3.gif ../results/materials_baseline/orbit.gif
python volume_rendering_main.py --config-name=nerf_materials +implicit_function.view_dependent=True training.resume=False training.checkpoint_path=checkpoints/materials_view.pth training.checkpoint_interval=249 training.render_interval=249 renderer.chunk_size=4096
cp images/part_3.gif ../results/materials_view/orbit.gif
cd ../GS
python unit_test_gaussians.py
python train.py --out_path output/truck
python render.py --out_path output/chair_dc
python render.py --sh --out_path output/chair_sh
cd ..
python evaluate.py --task nerf --config nerf/configs/nerf_lego.yaml --checkpoint nerf/checkpoints/lego_baseline.pth --output results/lego_baseline
python evaluate.py --task nerf --config nerf/configs/nerf_materials.yaml --checkpoint nerf/checkpoints/materials_baseline.pth --output results/materials_baseline
python evaluate.py --task nerf --config nerf/configs/nerf_materials.yaml --view-dependent --checkpoint nerf/checkpoints/materials_view.pth --output results/materials_view
python evaluate.py --task truck --checkpoint GS/output/truck/gaussians.pth --output results/truck
python make_figures.py --output ../proj3/assets