Volume Rendering, NeRF, and 3D Gaussian Splatting
Kaushik Chavali (kchavali)


Stratified samples between near and far along each ray:


Optimized box center and side lengths with MSE on rendered vs. GT colors (rounded to 1/100):
| x | y | z | |
|---|---|---|---|
| Box center | 0.25 | 0.25 | 0.00 |
| Side lengths | 2.01 | 1.50 | 1.50 |
Position is mapped with a harmonic embedding into an MLP (with a skip connection). Density uses a ReLU head so it stays non-negative; RGB uses a sigmoid. Trained on the lego scene at 128×128 for 250 epochs. The spiral below matches the expected bulldozer structure and colors.
Density is still a function of 3D position only. After the position MLP, the viewing direction is harmonically embedded and concatenated into a small color head, so only appearance can change with view. On the materials scene, this shows up on the shiny spheres: highlights shift as the camera moves, which a position-only color cannot represent.
Trade-off (view dependence vs generalization). More view dependence fits specular / glossy effects better because color can change with the ray direction. The downside is overfitting: if the direction branch is too strong, the network can memorize training-view colors instead of a consistent 3D appearance, which hurts novel views. Keeping density view-independent and restricting direction to a late, low-capacity color head limits that risk while still allowing useful specular variation.
I use 64 stratified coarse samples, evaluate a coarse NeRF to get weights along each ray, then draw 64 fine samples with inverse-CDF sampling from those weights. Coarse and fine depths are merged and sorted so the fine network sees 128 points. The training loss is fine-image MSE plus coarse-image MSE.
Trade-off (speed / quality). Hierarchical sampling improves quality by spending capacity near surfaces instead of empty space, which helps thin structure and sharper silhouettes for a fixed total sample count. The cost is speed and memory: every training step runs two networks (coarse then fine), so it is slower than a single uniform NeRF pass with the same number of samples. In short, better sample efficiency / image quality at the price of roughly ~2× network evaluation.
An MLP predicts an SDF (unbounded linear head) and a second MLP predicts RGB (sigmoid). Density uses the VolSDF Laplace CDF mapping σ = α Ψ_β(d), and training adds an eikonal term so ‖∇d‖ stays near 1.
What α and β do. α multiplies the whole density curve: it controls how opaque the medium is (peak density height) but does not move the SDF zero level set. β is the Laplace scale: it controls how wide the density band is around d = 0. Small β → a thin, sharp shell tightly around the surface. Large β → a thick, soft shell that spreads density farther from the zero set.
1. How does high β bias the learned SDF? Low β? High β spreads density over a thick band, so many points near the surface look “occupied.” That biases the SDF toward a blurrier / slabby shell (the zero set is less tightly constrained by the render). Low β concentrates density on a thin band, so the renderer mainly “sees” a sharp surface; the SDF is biased toward a tighter zero-crossing, but the density peak is narrower and easier to miss with sparse samples.
2. Easier to train with low β or high β? Why? High β is easier with volume rendering. A wider density band means more samples along each ray receive nonzero density and useful gradients. With low β, only samples very close to the surface contribute, so early training can stall if rays miss the thin shell.
3. More accurate surface with high β or low β? Why? Low β is more likely to learn an accurate surface. Because density is tightly tied to d ≈ 0, matching images pushes the zero level set to the true surface. High β can explain the same pixels with a thicker, softer SDF, so the recovered surface is less precise.
Settings chosen. I fixed α = 10 and eikonal weight = 0.02 (config defaults) and mainly tuned β. Best result: β = 0.05 — sharp enough geometry without being too hard to optimize. A much larger β (e.g. 0.5) made the under-object region overly filled / slabby, so I keep β = 0.05 as the reported run.


Splatting pretrained Gaussians with DC (view-independent) color only:
Isotropic Gaussians, means initialized from the truck point cloud, L1 loss, Adam.
| Parameter | Learning rate |
|---|---|
| Opacities | 0.05 |
| Scales | 0.005 |
| Colours | 0.025 |
| Means | 0.00016 |


Full SH (Q7.1) vs DC-only (Q5.5) on the same pretrained Gaussians. GIFs first, then same-view still frames (left: without SH, right: with SH).





Frame 0. Overall shape and silhouette match closely. With SH, the seat fabric shows a bit more shading variation; DC-only looks slightly flatter on the cushion.
Frame 10. Side / wood view: geometry is the same, but SH changes local brightness on the wood and fabric as the view direction changes. Differences are small because this chair is mostly diffuse.
Frame 20. Underside / back: again silhouettes align. SH adds mild view-dependent contrast on fabric regions; wood and underside change the least. Overall, SH helps appearance a little on this scene but does not dramatically change structure.
Materials scene with random initialization of 10k isotropic Gaussians (harder than Q6’s clean truck point cloud).
Baseline. Same recipe as Q6: isotropic Gaussians, L1 loss, same learning rates (means LR 0.00016, opacities 0.05, scales 0.005, colours 0.025), 1000 iterations.
Improved (modifications vs Q6 / baseline). Higher means learning rate (0.0005) and 2000 iterations so randomly placed Gaussians can move farther toward the objects. Anisotropic Gaussians were tried but ran out of GPU memory even with a small gaussians_per_splat, so the reported method stays isotropic with these LR / iteration changes.
| Means LR | Iters | PSNR | SSIM | |
|---|---|---|---|---|
| Baseline | 0.00016 | 1000 | 17.915 | 0.662 |
| Improved | 0.0005 | 2000 | 19.252 | 0.699 |
Improved gains about +1.34 dB PSNR and +0.037 SSIM. The progress GIFs show the baseline staying blurrier longer, while the improved run recovers the spheres and floor more cleanly by the end.



