Assignment 2
Jianing Zhang (jianing7)
1. Exploring loss functions (10 points)
1.1 Fitting a voxel grid (4 points)


1.2 Fitting a point cloud (3 points)


1.3 Fitting a mesh (3 points)


2. Reconstructing 3D from single view (80 points)
2.1 Image to voxel grid (20 points)










F1@0.05: 74.84
2.2 Image to point cloud (20 points)










F1@0.05: 79.48
2.3 Image to mesh (20 points)










F1@0.05: 72.89
Point cloud (79.48) > voxel (74.84) > mesh (72.89). The point decoder can place its points anywhere and F1 is measured on points, so it is the least constrained. The voxel grid is limited by its 323 resolution. The mesh is a deformed sphere, so it cannot open holes or represent thin, open parts well.
2.4 Analyse effects of hyperparams variations (10 points)
I varied w_smooth for the mesh decoder (20k steps each). Smoothness is the average Laplacian smoothness loss of the predicted meshes on test chairs (lower is smoother).
| w_smooth | 0.1 | 1 | 10 |
|---|---|---|---|
| F1@0.05 | 73.71 | 72.21 | 71.14 |
| smoothness | 1.66 | 0.33 | 0.067 |
More smoothing gives lower F1: the Laplacian term pulls vertices toward their neighbors and rounds off thin legs and edges. The surfaces look cleaner, but they are less accurate.










Each row is the same chair predicted with w_smooth 0.1, 1 and 10. At 0.1 the boxy armchair keeps its flat sides and corners but the surface is torn into folded triangles; at 10 it turns into a rounded, inflated shape and the thin legs of the ladder-back become thick tubes.
2.5 Interpret your model (10 points)
I made two visualizations, one for what the voxel model is unsure about and one for how much the input view matters.
1. Where is the voxel model unsure?
Predicted occupancy probability of the voxel model (middle slice along each axis) next to the ground truth. The model is confident on large flat parts and unsure on thin parts and boundaries.

2. Does the prediction depend on the input view?
The same chair from all of its views, through the point model. The predicted shape barely changes with the viewpoint, so how hard the chair is matters more than the view.


3. Exploring other architectures / datasets (choose at least one, more than one is extra credit)
3.1 Implicit network (10 points)
The decoder takes the image feature and a 3D coordinate and predicts the occupancy there. I query it on a 32x32x32 meshgrid in (-1, 1)3: each coordinate is concatenated with the 512-d feature and passed through a small shared MLP (128 wide, three residual blocks). 22 of 678 test chairs, mostly thin ones, have no voxel above the threshold and are scored 0.










F1@0.05: 72.62
Compared to the voxel decoder (74.84), it scores a little lower overall but handles thin shells better (test 300: 74.9 against 45.8), since one small network shared by all points learns a general rule for occupancy instead of 32,768 separate outputs. It also needed much longer training to get there.
3.2 Parametric network (10 points)
The decoder maps a random 2D point (u, v) in [0, 1]2, together with the image feature, to a 3D point, so it learns to fold a flat square onto the chair surface. One sheet has to cover legs, seat and back, so it gets stretched across the gaps between parts.










F1@0.05: 72.48
Compared to the point decoder (79.48), it scores lower: its points form one connected surface, which looks more regular, but a single sheet cannot cover separate parts like legs without leaving points in the gaps (test 150: 49.8 against 55.6).
3.3 Extended dataset for training (10 points)
Point cloud model trained on chairs only versus on chairs, cars and planes, evaluated on each class.
| test set | trained on chairs | trained on 3 classes |
|---|---|---|
| chair | 79.48 | 80.01 |
| car | 73.36 | 92.81 |
| plane | 66.50 | 96.21 |
Training on three classes does not hurt chairs (80.01 against 79.48), even though only about half of the training data is chairs. The chair-only model cannot represent other classes: for a plane it predicts a chair-sized cloud instead of thin wings.











