Assignment 2

16-825 Learning for 3D Vision

Task 1.1

Fit Vox Gif

Voxel: source; target

Task 1.2

Fit Point Gif

Point cloud: source; target

Task 1.3

Fit Mesh Gif

Mesh: source; target

Task 2.1

Example 0

Voxel example 0

RGB | pred | GT

Example 1

Voxel example 1

RGB | pred | GT

Example 2

Voxel example 2

RGB | pred | GT

Voxel F1 plot

F1

Task 2.2

Example 0

Point example 0

RGB | pred | GT

Example 1

Point example 1

RGB | pred | GT

Example 2

Point example 2

RGB | pred | GT

Point F1 plot

F1

Task 2.3

Example 0

Mesh example 0

RGB | pred | GT

Example 1

Mesh example 1

RGB | pred | GT

Example 2

Mesh example 2

RGB | pred | GT

Mesh F1 plot

F1

Quantitative comparison

Representation F1@0.05
Voxel grid 73.9
Point cloud 72
Mesh 69.7

The three models' performances are close. Voxel is the best, then point cloud and and then mesh. Voxel is the best because it's just a binary classification problem, which is easiest to train. The point cloud is the second best, because the points can move freely, making it harder. Mesh is the worst, because it deform an icosphere. The faces cannot change and only the vertices move. The smoothness loss may make the model miss some details.

Task 2.4. Abalation study: Number of points

F1@0.05: 66.2 (500), 71.4 (1000), 73.2 (1500). More points cover the surface better. When the number is over 1000, the improvement of more points is less significant.

Task 2.5

Deform

Interprete by starting the training from scratch, predict the same target every some steps, and visualize how the icospher is gradually deformed.

Task 3.1

Example 0

Implicit example 0

RGB | pred | GT

Example 1

Implicit example 1

RGB | pred | GT

Example 2

Implicit example 2

RGB | pred | GT

Implicit F1 plot

F1