Assignment 4 : Learning for 3D Vision

Abhinav Agarwalla (AndrewID: aa4)

Q1

The sphere tracing algorithm is run for a total of max_iters. At each iteration,

• Obtain the distance to the nearest surface by evaluating implicit_fn on origin points
• Ignore distances less than a threshold eps, say 1e^{-5}
• Increment origins by obtained distance in the direction directions
• Iterate until max_iters or if all rays reach a surface (checked by implicit_fn(origins) < eps)

Q2

Q3

Discussion on alpha and beta:

• alpha: determines the opacity of the object with respect to the material the object is made of. For example, a high alpha signifies that the object is very opaque and lets only a small proportion of light to travel through it.

• beta: determines the rate of drop in opacity on entering the object. A high beta would result in a smoother drop in transparency, while a lower beta would cause a sharp drop.

• How does high beta bias your learned SDF? What about low beta? Answer: A high beta spreads the surface of a learned SDF over a larger area i.e. increases its thickness with density close to alpha for these points. A low beta would result in sharp drops leading to thinner surfaces.

• Would an SDF be easier to train with volume rendering and low beta or high beta? Why? Answer: A low beta is prefered since otherwise sphere-tracing would result in noisy distances which could be far away from the actual surface. Moreover, the gradients would be less informative with high beta due to smoother drop in opacity.

• Would you be more likely to learn an accurate surface with high beta or low beta? Why? Answer: A low beta is prefered since otherwise a large area would constitute a surface. This means loosing out on details of the surface, leading to blob-y volume renderings.

Geometry	Color

Q4.1

Please use the config file composite.yaml for generating the below figure:

Q4.2

Number of Images	VolSDF	NeRF
5
10
20
50

VolSDF is able to render the LEGO bulldozer with as low as 10 images. The results from just 5 views are much worse, which might require more hyper-parameter tuning or careful selection of 5 most informative views.

Additionally, we can observe that VolSDF is better than NeRF when number of views is less than 50. At 50 views, NeRF starts performing better, capturing fine-grained details.

Please use config volsdf_nview.yaml with module a4.main_fewviews to get VolSDF results, and config nerf_lego_fewviews.yaml with module main_fewviews to obtain NeRF results.

Q4.3

Please use the config file volsdf_neus_sdf_to_dist.yaml for generating the below figure:

Geometry	Color