Volume Rendering, NeRF, and 3D Gaussian Splatting


A. Neural Volume Rendering

1.3. Ray sampling

grid rays

1.4. Point sampling

sample points

1.5. Volume rendering

part 1 depth

2. Optimizing a basic implicit volume

Center: (0.25, 0.25, -0.00) Side lengths: (2.01, 1.50, 1.50)

part 2

3. Optimizing a Neural Radiance Field (NeRF)

part 3

4.1 View Dependence

To make emission view-dependent, we concatenated the harmonic embedding of ray direction to position features, then pass into colour head. Density is kept position dependent only.

baseline (position only) view-dependent
materials baseline materials view

The view-dependent model places sharper specular highlights on the shiny spheres, but with few training views the model can overfit on those views instead of memorizing a consistent geometry. By keeping density posistion dependent only, geometry stays view-consistent and only emission varies.


B. 3D Gaussian Splatting

5.5 Perform Splatting

part 5

6 Training 3D Gaussian Representations

Learning rates: means 0.001, scales 0.005, colours 0.005, opacities 0.05

1000 iterations

PSNR: 30.12, SSIM: 0.946

part 6 progress part 6 final

7.1 Rendering Using Spherical Harmonics

Degree-3 real spherical harmonics evaluated per gaussian along the camera-to-gaussian direction (vectorized over all N gaussians).

Without SH v.s. with SH:

part 5 part 7.1 with SH

Side-by-side comparisons, left = DC only, right = full SH:

compare 1

Front view: the full-SH render recovers the gold damask pattern, while the DC-only render averages view-dependent highlights away.

compare 2

Top view: The full-SH render shows deeper contrast in colors and patterns, with brighter highlights on the gold trim.

compare 3

Underside view: This view is showing only diffuse and non-shiny surface, so the higher-order SH terms have only small contribution, and the two renderngs look similar.

7.2 Training On a Harder Scene

Baseline — Q6 setup with init_type="random":

PSNR: 19.94, SSIM: 0.702

7.2 baseline progress 7.2 baseline final

Improved:

PSNR: 20.79, SSIM: 0.770

7.2 improved progress 7.2 improved final

Modifications: