Learned Neural SDF
Point cloud used for training
I added view dependence to my NeRF implementation following the architecture describe in the original NeRF paper. After passing the 3D location through the encoder section of the architecture, I concatenated the view direction before passing it through the color output head.
I implemented sphere tracing by running a for loop where I update the depth of each ray based on the value of the SDF, and then update the current positions of the rays based on the new depth. I only update rays for which I have not yet reached the surface. I determine if I have reached the surface by checking if the absolute distance is below 1e-5
Architecture: I used a similar architecture to the one used in NeRF. I used harmomic positional encodings for the input positions. Then, I used an MLP with 6 layers and 128 hidden dimensions per layer. I used a skip connection between the input and the 4th layer and ReLU activations in between layers. Finally, I used a linear layer to output the SDF value (without any activation, to allow real value outputs).
Eikonal Loss: To implement the Eikonal loss, I used the MSE Loss between the norm of the gradients and the target value 1.
Alpha controls the level of density inside a volume. Meanwhile, beta controls how slowly the density decrease near the surface of a volume.
How does high beta bias your learned SDF? What about low beta?High beta makes the density decrease slowly near the surface. This would make the model learned more smooth surfaces. Meanwhile, low beta makes the density decrease quickly near the surface. This would make the model learn more sharp surfaces.
Would an SDF be easier to train with volume rendering and low beta or high beta? Why?It would be easier to train with a high beta because that makes more rays carry loss signals to the model. With a higher beta, if a ray travels close to the surface, it would still come across some density, and the gradient would encourage to move the surface closer to the ray.
Would you be more likely to learn an accurate surface with high beta or low beta? Why?Using a low beta will lead to learning more accurate surfaces because the density would not extend too far from the surface. This would encourage the model to learn surfaces that precisely match the images.
I implemented an alternative sdf_to_density function based on the sigmoid activation function. By taking the sigmoid of the negative signed distance function we can get a density that goes from 1 to 0 as we move away from the surface. To make the surfaces more sharp, we can multiply the signed distance by a scaling factor before passing it through the sigmoid. I used a scaling factor of 10.
The results were not as good as the original VolSDF implementation. It seems like the densities are not changing as sharply as in the original implementation, which is leading the model to learn a blurry image and a rounded shape. The other possible explanation is that the density should be a lot higher than 10 inside the volume. This could also explain why the model is learning a "cloudy" volume instead of a solid one.
