Assignment 3¶

Geethika Hemkumar (Andrew ID: ghemkuma)

1. Differentiable Volume Rendering¶

1.3 Ray Sampling¶

Grid Visualization¶

xy_grid_vis

Rays Visualization¶

rays_vis

1.4 Point Sampling¶

Visualization of Point Samples from Camera¶

visualization of point samples from camera

1.5 Volume Rendering¶

Visualization of Depth Map¶

depth map

GIF of Rendered Box¶

volume rendered box

2. Optimizing a Basic Implicit Volume¶

2.2 Optimized Box Statistics¶

Box Center: (0.25, 0.25, -0.00052)

Box Side Lengths: (2.01, 1.50, 1.50)

2.3 Optimized Box Visualization¶

optimized box implicit volume

3. View Independent NeRF¶

view independent nerf, nerf_lego scene

4. NeRF Extras¶

4.1 View Dependence¶

Visualizations¶

No description has been provided for this image
No View Dependence
No description has been provided for this image
With View Dependence

Static Frames¶

No View Dependence

With View Dependence

Absolute Difference

Frame 3

No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

Frame 11

No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

Frame 18

No description has been provided for this image
No description has been provided for this image
No description has been provided for this image

Differences between View Independent NeRF vs View Dependent NeRF¶

Though a little difficult to tell at a low resolution, the specularities on the spheres in the frames rendered with view dependence (middle column) are more localized (which indicates correct view-dependent rendering) compared to those in the frames rendered with no view dependence (left most column). This difference is highlighted by the difference maps (right most column), which are computed as an absolute difference in per-channel pixel values. The differences in frame 11 are most prominent.

Tradeoffs between increased view dependence and generalization quality¶

Increasing view dependent rendering improves the scene fidelity, as real-world effects such as specularities and reflections are modeled more realistically. However, as a result of learning these view-dependent effects, the generalization quality will decrease because the network is focusing on capturing lower-level scene details in the training dataset which are tied to specific camera views.

5. 3D Gaussian Rasterization¶

Chair rendered from multiple views with pre-trained Gaussians

6. Training 3D Gaussian Representations¶

Visualizations¶

Training Progress¶

Truck scene training progress GIF

Final Renders¶

Truck scene final render GIF

Hyperparameters¶

Learning rates:

pre_act_opacities: 0.01

pre_act_scales: 0.001

colours: 0.01

means: 0.001

Training iterations: 300

Metrics¶

PSNR: 28.226

SSIM: 0.926

7. Extensions¶

7.1 Rendering Using Spherical Harmonics¶

Comparison of GIFs Between Q5.5 and Q7.1¶

With View Dependent Spherical Harmonics (Q7.1)¶

chair scene with higher order SH

Only View Independent Spherical Harmonics (Q5.5)¶

chair scene with only 0th order SH

Side-by-Side RGB Renders Between Q5.5 and Q7.1¶

Q5.5

Q7.1

Frame 0

No description has been provided for this image
No description has been provided for this image

Frame 14

No description has been provided for this image
No description has been provided for this image

Frame 29

No description has been provided for this image
No description has been provided for this image

Analysis of Differences¶

Using higher order spherical harmonics helps to better model directional lighting effects. This improves the visibility of detail in the rendered chair, and the details that are more prominent in each frame depends on the camera viewpoint.

For frame 0 (first row), using higher order spherical harmonics (question 7.1, second column) highlights the details in the chair's seat fabric and its true (darker) green color. The detail on the arm rests is also shown more clearly.

For frame 14 (second row), the detail/color in the fabric on the chair's seat back is more prominent in the rendering using higher order spherical harmonics.

For frame 29 (third row), the rendering from question 7.1 shows a more prominent shadow compared to the rendering from question 5.5. This shadow corresponds to where the chair's seat back obstructs light from fully reaching the chair's seat.