16-825 Assignment 4

1. 3D Gaussian Splatting

1.1 3D Gaussian Rasterization

---

1.2 Training 3D Gaussian Representations

Training Progress

Final Render

Parameter	learning rate
opacities	0.001
scales	0.003
colours	0.02
means	0.01

Trained for 1000 iterations

Mean PSNR: 29.811

Mean SSIM: 0.939

---

1.3 Extensions

1.3.1 Rendering Using Spherical Harmonics

Original output	Spherical Harmonics Output

Original frame	View-dependent frame	difference
		view-dependent image has uniform shadow across the entire seat whereas the original frame has a bright strip that makes it look unnatural
		in the current orientation, the lower part of the seat in the view-dependent image looks a little brighter than the original image which maintains almost a similar colour from all prior frames
		the original image has a bright strip on half of the seat (making the lighting look inconsistent) but the view dependent image has the same shadow on the entire seat

---

2. Diffusion-guided Optimization

2.1 SDS Loss + Image Optimization

Prompt	Spherical Harmonics Output with guidance	Spherical Harmonics Output without guidance	iterations
a hamburger			2000
a standing corgi dog			2000
a ballerina			2000
a weightlifter with a dumbell			2000

---

2.2 Texture Map Optimization for Mesh

Prompt	Output
a purple and pink cow
a rainbow cow

---

2.3 NeRF Optimization

Prompt	Output	Depth
a standing corgi dog
a monkey
a flower

---

2.4 Extensions

2.4.1 View-dependent text embedding

Results are more consistent across view-points.

Prompt	Output	Depth	Comparison with Q2.3
a standing corgi dog			face of the dog remains consistent across views but in 2.3, we can see multiple front faces in the gif but here, we can see the front and back of the dog's head clearly
a flower			flower and stem shape and orientation remians consistent in different views here

---