Assignment 3: Neural Volume Rendering and Surface Rendering

Implementing NeRF and VolSDF - A Differentiable Rendering Pipeline

Part 0: Transmittance Calculation

Part A: Neural Volume Rendering

Part 1: Differentiable Volume Rendering

1.3 Ray Sampling

XY Grid
Pixel coordinates grid
Rays
Ray visualization

1.4 Point Sampling

Sample Points
Stratified sample points along rays

1.5 Volume Rendering

Part 1 Rendering
Spiral rendering of box SDF
Depth Map
Normalized depth map

Part 2: Optimizing a Basic Implicit Volume

Training Results

Before Training 0
Before training - View 0
Before Training 1
Before training - View 1
After Training 0
After training - View 0
After Training 1
After training - View 1
Part 2 Rendering
Optimized box - spiral rendering

Part 3: Optimizing a Neural Radiance Field (NeRF)

NeRF Results

Part 3 NeRF
NeRF trained on lego bulldozer (128x128, 250 epochs)

Part 4: NeRF with View Dependence

Comparison: Vanilla vs View-Dependent NeRF

Vanilla NeRF
Vanilla NeRF (position only)
View-Dependent NeRF
View-dependent NeRF

Part B: Neural Surface Rendering

Part 5: Sphere Tracing

Sphere Tracing Torus
Torus rendered using sphere tracing

Part 6: Optimizing a Neural SDF

Input Point Cloud
Input point cloud
Neural SDF Result
Learned neural SDF surface

Part 7: VolSDF

Understanding Alpha and Beta Parameters

The VolSDF paper introduces parameters alpha and beta in the SDF-to-density conversion formula. Consider the following questions:

  1. How does high beta bias your learned SDF? What about low beta?
      High beta biases the learned SDF by creating a smoother density distribution about the surface 0-crossing. Lower beta values theoretically create a more accurate surface representation as the density becomes an indicator function as beta approaches 0. (Concept taken from VolSDF paper, section 3.1)
  2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?
      Higher beta would be easier to train as it provides smoother gradients in comparison to a lower beta value which would have sharper gradients and be more prone to falling into local optima.
  3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?
      In theory, lower beta value would give a more accurate surface representation since as beta approaches 0, the SDF becomes an inidicator function at the object's surface. A higher beta value results in a smoother density, and the smoother this function the less accurate the surface is modeled. In practice, my results show something different. Higher beta values give more accurate surface reconstruction. My hypothesis is that higher beta values are more amenable to optimization (as beta is a learnable parameter), and thus are able to be optimized towards an accurate surface representation while lower beta values may fall into local optima with little chance of escape.

Parameter Exploration

Experimented with different alpha and beta parameters to understand their effect on the learned surface. The beta parameter controls the sharpness of the density transition around the surface, while alpha scales the overall density.

Alpha = 1.0, Beta = 0.05

Alpha 1.0 Beta 0.05 Geometry
Geometry
Alpha 1.0 Beta 0.05 Color
Colored surface

Alpha = 10.0, Beta = 0.05 (Default)

Alpha 10.0 Beta 0.05 Geometry
Geometry
Alpha 10.0 Beta 0.05 Color
Colored surface

Alpha = 10.0, Beta = 0.5

Alpha 10.0 Beta 0.5 Geometry
Geometry
Alpha 10.0 Beta 0.5 Color
Colored surface

Final VolSDF Results (Alpha = 5.0, Beta = 0.01)

These results use the optimal parameters found through experimentation: alpha = 5.0 and beta = 0.01. These values provide a good balance between surface accuracy and training stability.

VolSDF Geometry
Learned geometry
VolSDF Color
Colored surface

Part 8: Surface Extras

8.1 Render a Large Scene with Sphere Tracing

Created a complex scene with 20+ primitives (combination of spheres and boxes) distributed throughout 3D space.

Part 8 Complex Scene
Complex scene with 20 primitives rendered using sphere tracing

8.2 Fewer Training Views

Experimented with training VolSDF using only 20 training views instead of the full 100 views.

VolSDF Geometry with 20 Views
Learned geometry (20 training views)
VolSDF Color with 20 Views
Colored surface (20 training views)