1. Differentiable Volume Rendering
This part implements camera ray generation, stratified sampling along rays, and emission-absorption volume rendering.
Ray and sample visualization
Rendered color and depth
2. Optimizing a Basic Implicit Volume
Random pixel sampling and mean squared error were used to optimize the position and side lengths of an implicit box from posed training images.
Learned box parameters
| Box center (x, y, z) | (0.25, 0.25, 0.00) |
|---|---|
| Side lengths (x, y, z) | (2.01, 1.50, 1.50) |
3. Optimizing a Neural Radiance Field (NeRF)
A positional-encoding MLP predicts density and RGB color at sampled 3D positions. The network is trained with image supervision using mean squared error.
4. NeRF Extension - View Dependence
The view-dependent color branch uses viewing direction as an input to color prediction. This allows the model to represent directional appearance effects such as highlights and reflections.
View dependence makes the model more expressive because it can represent appearance changes caused by viewing direction, such as highlights and reflections. However, this added flexibility can also increase the risk of overfitting to the training views, potentially reducing generalization to unseen viewpoints. A view-independent model is simpler, while a view-dependent model can represent more complex appearance effects but may require sufficient training views to generalize well.
5. 3D Gaussian Rasterization
The rasterizer projects 3D Gaussian means and covariances into screen space, evaluates per-pixel alpha, computes front-to-back transmittance, and blends color, depth, and silhouette contributions.
6. Training 3D Gaussian Representations
Gaussians were optimized against posed training images using Adam and an L1 image loss.
Optimizer settings
| Parameter group | Learning rate |
|---|---|
| Gaussian means | 0.0001 |
| Gaussian scales | 0.005 |
| Gaussian colors | 0.01 |
| Gaussian opacities | 0.01 |
7.1. Spherical Harmonics
This section compares 3D Gaussian Splatting renderings with and without spherical harmonics. The rendering from Question 5.5 uses the original view-independent color representation, while Question 7.1 includes spherical harmonics for view-dependent color.
Rendering Comparison
RGB Image Comparisons
Each pair compares the same camera view rendered without and with spherical harmonics.
Comparison 1 - View 1
In this front view, the rendering with spherical harmonics has richer, darker green tones and more noticeable brown fabric details. Without spherical harmonics, the chair appears lighter and more uniform. The overall shape and structure remain nearly identical, and the depth and silhouette maps show no significant visible differences.
Comparison 2 - View 2
From this angled view, the rendering with spherical harmonics shows slightly richer colors and more contrast in the green fabric and brown decorative details. The rendering without spherical harmonics appears somewhat lighter. Both versions preserve the same chair geometry and viewpoint, with very similar depth and silhouette maps.
Comparison 3 - View 3
In this view, the two renderings are visually very similar. The chair retains the same overall structure and surface appearance, with only subtle differences in color and texture between the two versions. The depth and silhouette maps are also nearly identical, suggesting that the main visible differences come from color representation rather than changes in geometry.