4. NeRF Extras - View Dependence using nerf_materials.yaml
Trade off: Adding view dependence lets the NeRF model reproduce more realistic
effects like specular highlights and reflections that change with
the viewing angle, but it also makes the model more prone to overfitting
the training views. If the network relies too much on the viewing direction,
it can “memorize” how things look from seen cameras and fail to generalize
to novel viewpoints, producing artifacts or highlights that stick to the
camera. The key trade-off is between realism and generalization: more view
dependence captures shiny surfaces better but risks losing consistency and
stability across views.
B. Neural Surface Rendering
5. Sphere Tracing
In my implementation of the sphere_tracing function, I iteratively find where each
ray intersects the surface. Starting from the ray origins, I move each point forward
along its ray direction by the current SDF value. This process repeats until the SDF
becomes smaller than a small threshold, meaning the ray is close enough to the surface.
The loop stops once all rays have converged or the maximum number of iterations is reached.
The function returns the final intersection points and a boolean mask showing which rays
actually hit the surface. This makes the rendering both efficient and precise when locating surfaces.
6. Optimizing a Neural SDF
Input point cloudPrediction
The network architecture follows the NeRF implementation from Section 3, with a few adjustments.
Since the output represents a distance value, I removed the final ReLU layer used in NeRF.
All network parameters are initialized dynamically using the settings defined in the configuration (.yaml) file.
The Eikonal loss is calculated as the mean squared error between the L2 norm of the input gradients and 1.0,
enforcing the SDF to maintain a gradient magnitude of 1 everywhere.
7. VolSDF
Intuition for α and β: In VolSDF, β is a temperature/bandwidth that controls how “thick”
the volumetric shell around the zero-level set is. Low β → a very sharp, thin shell
(density spikes tightly around 𝑠(𝑥)=0); high β → a softer, thicker shell
(non-zero density spreads farther from the surface). α is a scale on extinction
(how opaque the shell is): higher α pushes rays to terminate more decisively near
the surface (stronger opacity/weights), while lower α makes everything more
transparent and gradients weaker.
Questions:
How does high beta bias your learned SDF? What about low beta?
For a high β: It blurs the surface into a thick volumetric band, biasing the SDF toward smoother, lower-frequency geometry and potentially underestimating sharp detail (highlights “floaters” if α is large). You get wider support and more forgiving gradients, but less crisp surfaces.h
For a low β: It concentrates density tightly around s=0, encouraging a crisp, thin surface close to the true level set. But gradients become peaky/sparse (vanish away from the surface), making optimization more fragile and sensitive to pose/noise.
Would an SDF be easier to train with volume rendering and low beta or high beta? Why?
High β is easier early on because it provides broader, non-zero gradients across space, so rays get useful supervision even when the surface isn’t well-located yet. Low β can stall if the surface guess is off.
Would you be more likely to learn an accurate surface with high beta or low beta? Why?
Low β yields the most accurate, crisp surface once the model is close, because density collapses onto the true level set. In practice I’d anneal: start with a higher β for stable gradients, then lower β (and tune α to avoid saturation) to sharpen the reconstruction.
