Assignment 3

A. Neural Volume Rendering (80 points)

0. Transmittance Calculation (10 points)

1. Differentiable Volume Rendering

1.3. Ray sampling (5 points)

Rays

xy Grid

1.4. Point sampling (5 points)

1.5. Volume rendering (20 points)

2. Optimizing a basic implicit volume

2.1. Random ray sampling (5 points)

2.2. Loss and training (5 points)

Box center: (0.25, 0.25, 0.00)
Box side lengths: (2.01, 1.50, 1.50)

2.3. Visualization

3. Optimizing a Neural Radiance Field (NeRF) (20 points)

4. NeRF Extras (CHOOSE ONE! More than one is extra credit)

4.2 Coarse/Fine Sampling (10 points)

This is actually slower to train but a little better in quality. You can actually tell the difference in quality at lower number of epochs where the course/fine sampling strategy has better quality off the bat but after many epochs they slowly start to converge. However, it is slower to use the course/fine strategy as there are more samples and points being trained on in each step.

B. Neural Surface Rendering (50 points)

5. Sphere Tracing (10 points)

We get the distance from a given set of points to the surface and take steps of size distance towards the steps iteratively from the origins of the ray. In the end, we see what points end up on the surface of the shape.

6. Optimizing a Neural SDF (15 points)

Input

Output

The MLP consists of 6 linear layers with ReLU actiavation followed by a linear layer that outputs a single value, which is the distance. The eikonal loss is calculated by finding how much the norm of the gradients deviate from 1.

7. VolSDF (15 points)

Questions:

1. How does high beta bias your learned SDF? What about low beta?
   A high beta biases the system to have more smoother gradual tranistion between inside and outside the surface while a low beta forces harder and sharper edges at the surface

2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?
   It would be easier to train with a high beta as the SDF would be more continuous and hence mosre stable gradients and easier optimization

3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?
   With a lower beta because the boundaries of the surface will have sharper transitions between inside and outside and so allows you to learn a more precise surface.

The settings I used was alpha: 10.0 and beta: 0.03. I reduced the beta a little from the initially given 0.05 to improve the sharpness of the edges.

8. Neural Surface Extras (CHOOSE ONE! More than one is extra credit)

8.2 Fewer Training Views (10 points)

I ran both the VolSDF solution and the NeRF solution with 20 views and the results are below

VolSDF

NeRF

From this, you actually see the opposite of what you expect theoretically, the NeRF is actually much better than the VolSDF.