HW 3 — Volume Rendering

A0 — Transmittance (Derivation)

T(t) = exp(-∫₀ᵗ σ(s) ds)
C = ∫ T(t) σ(t) c(t) dt
≈ Σ Tᵢ (1 - e^{-σᵢ Δᵢ}) cᵢ,  Tᵢ = Πⱼ<i (1 - αⱼ)
          

See the PDF for a careful discrete→continuous derivation and the piecewise solution for the figure with segments (y₁y₂), (y₂y₃), (y₃y₄).

A1 — Differentiable Volume Rendering (Box Scene)

αᵢ = 1 - exp(-σᵢ Δᵢ)
Tᵢ = Πⱼ<i (1 - αⱼ)
wᵢ = Tᵢ αᵢ
RGB  = Σ wᵢ cᵢ
Depth= Σ wᵢ tᵢ
        

A2 — Optimizing a Basic Implicit Volume (Box)

We randomly sample rays from each training image, render through the analytic box SDF with our differentiable volume renderer, and minimize MSE to the ground-truth colors. This inverts the renderer to recover the box parameters (center & side lengths) from images + known camera poses.

Learned Parameters

after 1,000 iters

Center (x, y, z)

(+0.25, +0.25, +0.00)

Side lengths (x, y, z)

(2.01, 1.50, 1.50)

Config: T4 • CUDA 12.1 • Torch 2.4 • PyTorch3D OK • batch=cfg.training.batch_size

Before training

After training

    # rays: random subset of pixels per iter (saves memory)
    # forward: sample points → densities/colors → transmittance weights → RGB
    # loss: MSE(pred_rgb, rgb_gt), backprop into box center & side lengths
    # outputs: part_2_before_training_*.png, part_2_after_training_*.png, images/part_2.gif
        

A3 — Neural Radiance Field (Lego)

View-independent NeRF trained on 128×128 Lego views. The MLP maps 3D position → density (ReLU) and color (Sigmoid), with positional encoding (L = 6). We render by integrating along rays using transmittance weights.

Ablation Analysis

• Positional Encoding Depth (L = 4 vs L = 6): Lower-frequency encoding (L = 4) blurs fine details—metallic wheels lose sharpness and the red markers dim. L = 6 captures higher-frequency geometry with cleaner silhouettes and brighter highlights. Measured difference: PSNR ≈ 33 dB vs. the L = 6 baseline (lower is worse).
• Samples per Ray (64 vs 128): Visual and PSNR differences are negligible; both converge to nearly identical reconstructions (PSNR ≈ 120 dB, MSE ≈ 0 when comparing spiral frames). For this smooth Lego scene, 64 samples per ray are already sufficient, so using fewer samples mostly improves efficiency without sacrificing quality.

Takeaway: Frequency encoding (L) drives spatial fidelity; sample count mainly affects compute for this dataset.

Training summary

250 epochs • batch 1024 • 128 samples/ray

• Positional Encoding: L=6 (xyz), view-independent colors
• Renderer: alpha = 1−exp(−σΔ), transmittance cumprod, weighted color sum
• Optimizer: Adam, lr 5e-4, step LR scheduler (γ=0.8 / 50 epochs)

PSNR (L=4)

33 dB

PSNR (samples/ray=64)

120 dB

Checkpoint

./checkpoints

Reproduce

    > conda activate learning3d

    # Main:
    > python volume_rendering_main.py --config-name=nerf_lego

    # Ablation 1 (L=4):
    > python volume_rendering_main.py --config-name=nerf_lego implicit_function.n_harmonic_functions_xyz=4 training.resume=False training.checkpoint_path=checkpoints_ablations/nerf_L4.pt

    # Ablation 2 (SPR=64)
    > python volume_rendering_main.py --config-name=nerf_lego sampler.n_pts_per_ray=64 training.resume=False training.checkpoint_path=checkpoints_ablations2/nerf_64spp.pt
            

Ablations (optional)

Compare PE level and samples/ray:

Part 4 — NeRF Extras

4.1 View-Dependent NeRF (Materials)

We extend NeRF to model view-dependent emission by conditioning the color head on the viewing direction (harmonic encoding with L_dir=2). Density depends only on position. This captures specular highlights and glossy BRDF effects.

• Config: n_harmonic_functions_dir = 2, xyz = 6; batch 1024; 128 samples/ray
• Trade-offs: +Realism for glossy materials; −Slightly more compute; risk of baked-in highlights if overfit

Part 4.2 — Coarse / Fine Sampling

We implemented hierarchical sampling: a coarse pass renders with uniform depths to estimate geometry, then a fine pass importance-samples additional depths from the coarse weights (PDF) and re-renders.

Observation

In our setup the hierarchical version appears slightly blurrier than the uniform baseline. This can happen when the coarse network’s density weights are not yet accurate, so the importance sampler spreads fine samples into suboptimal regions. At 128×128 and limited training, uniform sampling remained sharper and more stable.

Why this can happen

• Diffuse coarse weights: Early in training the PDF is broad → fine samples don’t focus on the surface.
• Low resolution / short training: Coarse geometry stabilizes late; supervising fine too early adds noise.
• Budget split: With the same total samples, moving some from uniform → PDF can reduce coverage.

Settings we used

Coarse

64

Fine

+64

Resolution

128×128

Tips that usually improve coarse/fine

• Warm-up with uniform only for a few epochs, then enable fine sampling.
• Use more total samples (e.g., 64 coarse + 128 fine) if memory allows.
• Build the PDF from midpoint bins and clip weights with a small ε to avoid spiky CDFs.
• Slightly reduce LR or add density noise early to stabilize the coarse field.

Part 5 — Sphere Tracing an SDF (Torus)

We render a signed–distance field by sphere tracing. From each ray origin we step forward by the current SDF value until the absolute distance falls below a small threshold (hit) or the ray exceeds far (miss). The renderer outputs the intersection point and a hit mask, then shades the hit location.

Part 6 — Optimizing a Neural SDF (with Eikonal)

We train an MLP to approximate a signed distance field (SDF) from a sparse point cloud. The network predicts distance at arbitrary 3D locations and is regularized by the eikonal loss, which encourages a unit-norm gradient (‖∇f(x)‖≈1) so the learned function behaves like a true distance field.