16-825 Learning for 3D Vision

Assignment 4

Rodrigo Lopes Catto | rlopesca

3D Gaussian Splatting

1.1.5 Perform Splatting

1.2 Training 3D Gaussian Representations

Number of iterations: 250

Learning rate parameters:

• opacities: lr- 0.005
• scales: lr - 0.01
• colours: lr - 0.02
• means: lr - 0.001

The values for PSNR & SSIM are as follows:

• Mean PSNR: 28.516
• Mean SSIM: 0.924

Training progress

Final rendered GIF

1.3.1 Rendering Using Spherical Harmonics

Frame	Without Spherical Harmonics	With Spherical Harmonics	Differences
Comparison 1			Lighting appears more realistic with spherical harmonics, showing softer shadows and richer texture detail on the seat.
Comparison 2			Spherical harmonics add shading variation and highlight depth, making materials look less flat and more natural.

2. Diffusion-guided Optimization

2.1 SDS Loss + Image Optimization

All the models below were trained for 2000 iterations.

'A hamburger'

Without Guidance	With Guidance

'A standing corgi dog'

Without Guidance	With Guidance

'a roller coaster'

Without Guidance	With Guidance

'f1 car'

Without Guidance	With Guidance

2.2 Texture Map Optimization for Mesh

Note: the gifs that are saved are not on continuous loop. Please refresh the webpage to start the gif video.

Prompt: 'Cow with tiger skin'

Prompt: 'Black and white cow'

2.3 NeRF Optimization

Parameters:

• lambda_entropy: 0.0001
• lambda_orient: 0.01
• latent_iter_ratio: 0.1

'a standing corgi dog' - 10,000 iterations

RGB	Depth

'a dinossaur' - 6000 iterations

RGB	Depth

'a duck' - 10000 iterations

RGB	Depth

Extensions

2.4.1 View-dependent text embedding

Parameters:

• lambda_entropy: 0.0001
• lambda_orient: 0.01
• latent_iter_ratio: 0.1

'a standing corgi dog' - 5000 iterations

RGB	Depth

Comparing this result with the corgi generated without view dependence, the main difference is the dog’s pose, as it is standing in the previous result and sitting here. In addition, the view-dependent model achieved a comparable or slightly better level of structural detail, with the nose now visible, while using only half the number of iterations of the non-view-dependent one.

'a dinossaur' - 2000 iterations

RGB	Depth

The dinosaur rendered with view dependence shows a more coherent structure and better-defined silhouette, particularly around the head and tail regions. Despite being trained for only 2000 iterations, it already achieves a recognizable shape and some texture surface compared to the non-view-dependent version, which after 6000 iterations still appears flatter and less consistent in geometry.