Assignment 3 : Neural Volume Rendering and Surface Rendering

Trade-offs between View Dependence and Generalization:

Increased view dependence allows NeRF to model specular reflections and view-dependent effects, producing more photorealistic renderings of glossy materials. However, this comes at the cost of requiring more training views to generalize well, as the model can overfit to specific viewpoints and produce artifacts on novel views.

I implemented sphere tracing in renderer.py using an iterative ray marching algorithm. The method starts each ray at the near plane and iteratively advances along the ray by the absolute value of the SDF at the current point, which represents the safe distance to march without overshooting the surface. I maintain three states for each ray: converged (rays that hit the surface within threshold 1e-4), escaped (rays beyond the far plane), and active (rays still marching). The algorithm terminates when all rays have either converged or escaped, or after reaching max_iters iterations. The output includes intersection points and a boolean mask indicating which rays successfully hit the surface.

The left is the input and the right is the training result.

MLP Architecture: I implemented a NeuralSurface MLP in implicit.py with two separate branches. The distance branch uses positional encoding on xyz coordinates, processes them through 6 hidden layers (128 neurons) with ReLU activations and skip connections, then outputs a scalar signed distance value. The color branch takes the distance features and processes them through 2 additional layers (128 neurons) to predict RGB colors.

Eikonal Loss: Implemented in losses.py, the eikonal loss enforces that SDF gradients have unit norm everywhere: ||∇f(x)|| = 1.

How does beta bias your learned SDF?

High beta creates a smooth, gradual density transition producing a "soft" surface. Low beta creates a sharp transition concentrating density near the true surface, producing a "crisp" surface.

Easier to train with low or high beta?

High beta is easier to train—smooth gradients provide better gradient flow and tolerate inaccurate SDF values. Low beta is harder because the sharp sigmoid creates sparse gradients that only activate near the surface.

More accurate surface with high or low beta?

Low beta learns more accurate surfaces by forcing precise zero-level set localization. High beta allows the network to spread density over large regions, fitting images but producing geometrically fuzzy surfaces.

With only 20 training views, VolSDF produces better geometry and novel view synthesis than NeRF. VolSDF's eikonal regularization enforces smooth, well-behaved surfaces that generalize well from sparse observations. NeRF lacks geometric constraints and tends to overfit training views, producing noisy geometry and poor novel view quality. Surface-based representations are more data-efficient for sparse view reconstruction in this case.