16-825: Learning for 3D Vision - Assignment 3

Vaibhav Parekh | Fall 2025

A. Neural Volume Rendering

0. Transmittance Calculation

1.3. Ray sampling

Fig. 1.3(a). Grid
Fig. 1.3(b). Rays

1.4. Point sampling

Fig. 1.4. Point samples

1.5. Volume rendering

Fig. 1.5(a). Volume rendering
Fig. 1.5(b). Depth

2.2. Loss and training

Center of the box after training: (0.25, 0.25, 0.00)

Side lengths of the box after training: (2.00, 1.50, 1.50)

Fig. 2.2. Center and the Side Lengths of the box after training

2.3. Optimizing a basic implicit volume - Visualization

Fig. 2.3(a). Visualization

Fig. 2.3(b). Before Training

Fig. 2.3(c). After Training

3. Optimizing a Neural Radiance Field (NeRF) - Visualization

Fig. 3. NeRF Lego visualization

4.1. NeRF Extras - View Dependence

Fig. 4.1. Nerf Materials visualization

Trade-offs between View Dependence and Generalization

To achieve a good balance, one should include a wide range of camera poses during training and use regularization or shared feature constraints to limit overfitting.

B. Neural Surface Rendering

5. Sphere Tracing

Fig. 5. Sphere Tracing

Sphere tracing advances rays through the scene by repeatedly stepping forward by the signed distance to the nearest surface. At each step, the implicit function estimates how far the current point is from the surface. When this distance falls below a small threshold (epsilon), the ray is considered to have hit the surface. Rays that never reach this threshold within the iteration limit are marked as misses.

6. Optimizing a Neural SDF

Fig. 6(a). Input point cloud used for training
Fig. 6(b). Prediction

The MLP uses a single network with a skip connection at the fourth layer. Unlike NeRF, its distance head ends with a linear layer since SDF values are continuous and unbounded. The eikonal loss enforces that the gradient magnitude equals one by minimizing the mean squared error between ‖∇f(x)‖ and 1, ensuring the network learns a valid signed distance field.

7. VolSDF

Fig. 7(a). Geometry
Fig. 7(b). Color

Intuitive explanation of alpha and beta

Q1. How does high beta bias your learned SDF? What about low beta?
A high beta pushes the model to form a very sharp density boundary around the surface, which can improve precision but also make training unstable and prone to noise or overfitting. A low beta smooths the density transition, making optimization easier and more stable, but may blur the surface and reduce geometric accuracy.

Q2. Would an SDF be easier to train with volume rendering and low beta or high beta? Why?
Training is generally easier with a lower beta, since gradients are smoother and less abrupt. The optimization landscape becomes simpler, leading to faster convergence. In contrast, high beta increases gradient sensitivity near the surface, making the training process harder and less stable.

Q3. Would you be more likely to learn an accurate surface with high beta or low beta? Why?
A higher beta produces sharper and more well-defined surfaces, making it better for capturing fine geometric detail once training has stabilized. However, it is harder to optimize and may introduce artifacts if used too early. A lower beta helps achieve smoother convergence but may result in slightly blurred or less precise surfaces. In practice, accuracy is often improved by starting with a low beta and gradually increasing it during training.

8.2. Neural Surface Extras - Fewer Training Views

Fig. 8.2(1-a). NeRF (20 views)
Fig. 8.2(1-b). Geometry (20 views)
Fig. 8.2(1-c). Color (20 views)
Fig. 8.2(2-a). NeRF (40 views)
Fig. 8.2(2-b). Geometry (40 views)
Fig. 8.2(2-c). Color (40 views)
Fig. 8.2(3-a). NeRF (60 views)
Fig. 8.2(3-b). Geometry (60 views)
Fig. 8.2(3-c). Color (60 views)
Fig. 8.2(4-a). NeRF (80 views)
Fig. 8.2(4-b). Geometry (80 views)
Fig. 8.2(4-c). Color (80 views)
Fig. 8.2(5-a). NeRF (100 views)
Fig. 8.2(5-b). Geometry (100 views)
Fig. 8.2(5-c). Color (100 views)