Assignment 3: Volume Rendering, NeRF, and 3D Gaussian Splatting¶
Name: Chenzhe Zhang
Course: 16-825 Learning for 3D Vision
1.3 Ray Sampling¶
| Pixel Coordinates | Ray Directions |
|---|---|
![]() |
![]() |
1.4 Point Sampling¶

1.5 Volume Rendering¶
| Volume Rendering | Normalized Depth |
|---|---|
![]() |
![]() |
2. Optimizing a Basic Implicit Volume¶
2.1 Random Ray Sampling¶
I randomly sample 4,096 pixels without replacement from each image and generate their corresponding rays for training to reduce GPU memory usage.
2.2 Loss and Training¶
| Parameter | x | y | z |
|---|---|---|---|
| Center | 0.25 | 0.25 | 0.00 |
| Side lengths | 2.01 | 1.50 | 1.50 |
2.3 Visualization¶

3. Neural Radiance Field¶

I trained a position-encoded MLP with six hidden layers of 128 units for 250 epochs at a resolution of 128 × 128.
4.1 View Dependence¶

Density depends only on position, while RGB also uses the encoded viewing direction. This allows the model to represent view-dependent reflections, but excessive directional capacity can overfit the training views and reduce generalization to unseen views. I use a small color branch with 64 hidden units and train for 250 epochs at a resolution of 128 × 128.
5. 3D Gaussian Rasterization¶
5.1 Projecting 3D Gaussians¶
I compute the 3D covariance from each Gaussian's scale and rotation, then project it to 2D using the camera rotation and the projection Jacobian. Gaussian centers are projected to pixel coordinates.
5.2 Evaluating 2D Gaussians¶
I compute the Gaussian power at each pixel using its offset from the projected mean and the inverse 2D covariance.
All four provided unit tests passed: 4/4.
5.3 Filtering and Sorting Gaussians¶
I transform Gaussian centers into camera space, discard Gaussians with non-positive depth, and sort the remaining Gaussians from front to back.
5.4 Alphas and Transmittance¶
I compute per-pixel alpha by multiplying each Gaussian's opacity by its Gaussian response. Transmittance is the product of $(1-\alpha)$ over all preceding Gaussians and is carried across mini-batches.
6. Training 3D Gaussian Representations¶
6.1 Parameters and Optimizer¶
I optimize 10,000 isotropic Gaussians using Adam with the following learning rates.
| Parameter | Learning rate |
|---|---|
| Means | 0.0001 |
| Scales | 0.005 |
| Opacities | 0.05 |
| Colors | 0.01 |
7. Gaussian Splatting Extension¶
7.1 Rendering Using Spherical Harmonics¶
I use spherical harmonics up to degree 3 to compute view-dependent RGB colors.
DC Only¶

Full Spherical Harmonics¶

Same-View RGB Comparisons¶
Left: DC only. Right: full spherical harmonics.
Frame 000

Full SH produces darker green upholstery and stronger contrast in the gold pattern.
Frame 005

Full SH changes the brightness across the seat and backrest, while the silhouette remains unchanged.
Frame 020

The difference is smaller on the pale underside. Similar streaking artifacts remain around the legs in both versions.





