16-825 Assignment 4¶

1. 3D Gaussian Splatting¶

1.1 3D Gaussian Rasterization (35 points)¶

Render Output

1.2 Training 3D Gaussian Representations (15 points)¶

Parameter Learning Rate (lr)
opacities 0.0008
scales 0.001
colours 0.025
means 0.00015
Metric Value
Iterations 1000
Mean PSNR 27.843
Mean SSIM 0.920
Final Renders GIF
Training Progress GIF

1.3 Extensions (Choose at least one! More than one is extra credit)¶

1.3.1 Rendering Using Spherical Harmonics (10 Points)¶

Old Output

Old Render Output

New Output

New Render Output

📸 Rendering Comparison: New (Q1) vs. Old (Q1.1.5)¶

New Rendering (Q1/output/q1_render/) Old Rendering (Q1/output/q1.1.5_render/)
Frame 000 Frame 000
New Render Frame 000 Old Render Frame 000
Frame 018 Frame 018
New Render Frame 018 Old Render Frame 018
Frame 029 Frame 029
New Render Frame 029 Old Render Frame 029

The most prominent difference is the increased brightness of the newer rendering, which results in the couch shifting to a much lighter green hue when contrasted with the darker tones of the older rendering.¶

2. Diffusion-guided Optimization¶

2.1 SDS Loss + Image Optimization (20 points)¶

🖼️ Image Generation Comparison¶

Prompt Original Output (e.g., Iteration Output) SDS Output (e.g., sds_1)
1. A Hamburger Original Hamburger SDS Hamburger
2. A Pokemon flying Original Pokemon SDS Pokemon
3. A standing Corgi Dog Original Corgi SDS Corgi
4. Luffy Gear 5 fighting Original Luffy SDS Luffy

(All of the prompts were trained for 2k iterations)¶

2.2 Texture Map Optimization for Mesh (15 points)¶

🌐 3D Mesh Generation Comparison

Original Mesh (Reference) Prompt 1: A Hamburger Prompt 2: Disco Lights Prompt 3: Multi Coloured Lego
Original Mesh Hamburger Mesh Disco Lights Mesh Multi Coloured Lego Mesh

2.3 NeRF Optimization (15 points)¶

1️⃣ Prompt: A standing corgi dog 🐕

Example Image RGB Video Depth Video
Corgi RGB Image Your browser does not support the video tag. Your browser does not support the video tag.

2️⃣ Prompt: A red racing car 🏎️

Example Image RGB Video Depth Video
Racing Car RGB Image Your browser does not support the video tag. Your browser does not support the video tag.

3️⃣ Experimental Prompt: A f1 car on track (Dosent Work well)

Example Image RGB Video Depth Video
F1 Car RGB Image Your browser does not support the video tag. Your browser does not support the video tag.

2.4 Extensions (Choose at least one! More than one is extra credit)¶

2.4.3 Variation of implementation of SDS loss (10 points)¶

These results were implemented using L2 Loss¶

1️⃣ Prompt: A standing corgi dog 🐕

Example Image RGB Video Depth Video
Corgi RGB Image Your browser does not support the video tag. Your browser does not support the video tag.

2️⃣ Prompt: A red racing car 🏎️

Example Image RGB Video Depth Video
Racing Car RGB Image Your browser does not support the video tag. Your browser does not support the video tag.

💡 Loss Model Comparison Summary¶

Loss Model Training Details (10,000 Iterations) Key Observations
SDS Loss Trained for 10,000 iterations. Produced cleaner outputs with significantly less noise.
Pixel Space Loss Trained for 10,000 iterations. Slightly slower per iteration and appears to require more training iterations to achieve results comparable to the SDS model's output quality.

Conclusion¶

The SDS Loss Model provided superior visual quality within the same training budget (10,000 iterations), yielding outputs that were much cleaner and less noisy than the Pixel Space Loss Model. The Pixel Space Loss Model is likely underexposed and needs further training to reach comparable results.