Question 1

Exploring Loss Functions

Prediction
Ground truth

1.1 Voxel grid

Fitted voxel gridGround-truth voxel grid

1.2 Point cloud

Fitted point cloudGround-truth point cloud

1.3 Mesh

Fitted meshGround-truth mesh

Question 2.1

Image to Voxel Grid

68.50F1@0.05

Trained with pos_weight=3 in the BCE loss.

Voxel F1 vs threshold
Input
Prediction
Ground truth
Input 1Voxel prediction 1Ground truth 1 Input 2Voxel prediction 2Ground truth 2 Input 3Voxel prediction 3Ground truth 3

Question 2.2

Image to Point Cloud

77.59F1@0.05

Point cloud F1 vs threshold
Input
Prediction
Ground truth
Input 1Point prediction 1Ground truth 1 Input 2Point prediction 2Ground truth 2 Input 3Point prediction 3Ground truth 3

Question 2.3

Image to Mesh

72.62F1@0.05

Mesh F1 vs threshold
Input
Prediction
Ground truth
Input 1Mesh prediction 1Ground truth 1 Input 2Mesh prediction 2Ground truth 2 Input 3Mesh prediction 3Ground truth 3

Voxel vs. point cloud vs. mesh

RepresentationF1@0.05
Voxel68.50
Point cloud77.59
Mesh72.62

Point cloud is best because each point can move freely to the surface. The mesh is limited by the sphere's fixed topology (it cannot open holes between legs or slats). Voxels are limited by the 32³ resolution, so thin parts are lost or thickened.

Question 2.4

Effect of the Number of Points

I changed only n_points of the point cloud decoder.

n_points50010002000500010000
F1@0.0571.6877.5980.3383.7684.78
Input
500
1000
2000
5000
10000
Input 500 points 1000 points 2000 points 5000 points 10000 points

More points give a higher F1 and a more complete chair, but the gain gets smaller as n grows.

Question 2.5

Interpret the Model

I used the point cloud model for this since it had the best F1.

Input
Colored by index
Colored by error
Ground truth
Input 1Index-colored prediction 1Error-colored prediction 1Ground truth 1 Input 2Index-colored prediction 2Error-colored prediction 2Ground truth 2 Input 3Index-colored prediction 3Error-colored prediction 3Ground truth 3
Error color scale: distance to the ground-truth surface

Point correspondence

I gave each of the 1000 output points a color based on where it sits in the model's average prediction, and used the same colors for every test chair. The same colors end up on the same parts of every chair, so each output point has a fixed job, like being part of the front-left leg. Even the chair with no legs (middle) still gets the leg-colored points, placed along its bottom.

Error heatmap

Each point is colored by its distance to the ground truth. The normal chair is accurate; the errors are on the tilted backrest and the bottom of the chair with no legs.

Question 3.1

Implicit Network

67.59F1@0.05

Trained with pos_weight=3 in the BCE loss.

Implicit network F1 vs threshold
Input
Prediction
Ground truth
Input 1Implicit prediction 1Ground truth 1 Input 2Implicit prediction 2Ground truth 2 Input 3Implicit prediction 3Ground truth 3

Almost the same F1 as the voxel decoder (68.50) with 41× fewer decoder parameters (0.12M vs. 4.9M), since one small function is shared by all locations.

Question 3.2

Parametric Network

73.80F1@0.05

Parametric network F1 vs threshold
Input
Prediction
Ground truth
Input 1Parametric prediction 1Ground truth 1 Input 2Parametric prediction 2Ground truth 2 Input 3Parametric prediction 3Ground truth 3

3.8 F1 below the point decoder (77.59) with 8× fewer parameters (1.06M vs. 8.8M). Points are colored by patch: the patches overlap across the whole chair instead of splitting it into parts.

Question 3.3

Extended Dataset

Trained onChairCarAirplaneAll
Chair only76.5466.0962.9870.11
Chair + car + airplane77.0592.7795.5886.21
Input
Chair-only model
3-class model
Ground truth
Airplane inputChair-only model on an airplane3-class model on an airplaneAirplane ground truth Car inputChair-only model on a car3-class model on a carCar ground truth Chair inputChair-only model on a chair3-class model on a chairChair ground truth
F1 vs threshold of the chair-only and 3-class models on the 3-class test set

On the same chairs, adding two classes barely changes F1 (76.54 → 77.05). The chair-only model turns every input into a chair, so its outputs have little diversity. The 3-class model is consistent within a class and switches to a car or airplane shape from the image. Cars and airplanes score much higher than chairs, likely because they vary less within the class.