Volume Rendering, NeRF, and 3D Gaussian Splatting

1. Differentiable Volume Rendering

1.3 Ray sampling

xy_grid
rays

1.4 Point sampling

sample points

1.5 Volume rendering

part_1.gif
depth

2. Optimizing a Basic Implicit Volume

Box center: (0.25, 0.25, 0.00)

Box side lengths: (2.01, 1.50, 1.50)

part_2.gif

3. Optimizing a Neural Radiance Field (NeRF)

part_3.gif

4.2 Coarse/Fine Sampling

Trade-offs (speed / quality): the fine pass adds a second network and 192 samples per ray instead of 64, so each iteration is slower and uses more memory. Quality improves because the extra samples concentrate on surfaces instead of empty space, which recovers thinner geometry than uniform sampling.

part_4_2.gif

5. 3D Gaussian Rasterization

part_5_render.gif

6. Training 3D Gaussian Representations

Learning rates

ParameterLearning rate
opacities0.05
scales0.005
colours0.0025
means0.00016

Number of iterations: 1000

PSNR: 29.308

SSIM: 0.937

part_6_training_progress.gif
part_6_training_final_renders.gif

7.1 Rendering Using Spherical Harmonics

5.5 GIF
7.1 GIF

Frame 000

5.5 frame 000
7.1 frame 000

Spherical harmonics make the seat lighting more even and smoother.

Frame 010

5.5 frame 010
7.1 frame 010

Like above, the spherical harmonics make the seat lighting more even and smoother, the gold pattern on the seat is a bit more sharp.

Frame 020

5.5 frame 020
7.1 frame 020

Looks pretty similar, the spherical harmonics make the gold pattern on the seat a bit more clear.