Assignment 3 : Neural Volume Rendering and Surface Rendering

Goals: In this assignment, you will setup a differentiable rendering pipeline and implement neural volume/surface rendering techniques like NeRF and VolSDF.

Table of Contents

• Setup
• A. Neural Volume Rendering (80 points)
  • 0. Transmittance Calculation (10)
  • 1. Differentiable Volume Rendering (30)
  • 2. Optimizing a Basic Implicit Volume (10)
  • 3. Optimizing a Neural Radiance Field (NeRF) (20)
  • 4. NeRF Extras (10 + 10 Extra)
• B. Neural Surface Rendering (50 points)
  • 5. Sphere Tracing (10)
  • 6. Optimizing a Neural SDF (15)
  • 7. VolSDF (15)
  • 8. Neural Surface Extras (10 + 20 Extra)

A. Neural Volume Rendering (80 points)

0. Transmittance Calculation (10 points)

1. Differentiable Volume Rendering

1.3. Ray sampling (5 points)

1.4. Point sampling (5 points)

1.5. Volume rendering (20 points)

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

2.2. Loss and training (5 points)

2.3. Visualization

3. Optimizing a Neural Radiance Field (NeRF) (20 points)

Implementation

4. NeRF Extras (CHOOSE ONE! More than one is extra credit)

4.1 View Dependence (10 points)

B. Neural Surface Rendering (50 points)

5. Sphere Tracing (10 points)

6. Optimizing a Neural SDF (15 points)

7. VolSDF (15 points)

An intuitive explanation of what the parameters alpha and beta are doing here:

Ans: alpha controls the constant baseline density before any smoothing near the surface. beta controls the mean absolute deviation of the CDF of the laplace function, which determines how sharp of the density drop-off is near the surface.

1. How does high beta bias your learned SDF? What about low beta? High beta makes the surface drop-off less sharp, hense make the render less optimal. Low beta can make the model overly sensitive to noise and produce holes in the output.

2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why? High beta allows the model some room for error, and can be benificial for robust rendering.

3. Would you be more likely to learn an accurate surface with high beta or low beta? Why? An acurate render requires low beta due to the shaper transition on surfaces, leading to more accurate surfaces.

8. Neural Surface Extras (CHOOSE ONE! More than one is extra credit)

8.1. Render a Large Scene with Sphere Tracing (10 points)

40 randomly placed and randomly rotated toruses, spheres, box.