16-889 Assignment 2: Single View to 3D
Jianchun Chen jianchuc@andrw.cmu.edu
1. Exploring loss functions
1.1. Fitting a voxel grid (5 points)
Left: fitted voxel grid. Right: ground truth voxel grid
1.2. Fitting a point cloud (10 points)
Left: fitted point cloud. Right: ground truth point cloud
1.3. Fitting a mesh (5 points)
Left: fitted mesh. Right: ground truth mesh
2. Reconstructing 3D from single view
2.1. Image to voxel grid (15 points)
2.2. Image to point cloud (15 points)
1.3. Fitting a mesh (5 points)
2.4. Quantitative comparisions(10 points)
| \ | F-score@0.05 |
|---|---|
| Voxel | 78.76 |
| Point cloud | 91.85 |
| Mesh | 85.29 |
2.5. Analyse effects of hyperparms variations (10 points)
I study the hyperparameter n_points for mesh generation.
The results below show that during training, using chamfer distance over 500 and 5000 randomly sampled points gives similar performance for predicting mesh with ~2500 vertices.
| n_points | F-score@0.05 |
|---|---|
| 500 | 78.86 |
| 5000 | 85.29 |
The top images shows prediction trained with n_points=500 and the bottom images shows prediction trained with n_points=5000.
2.6. Interpret your model (15 points)
Here I trace and color the first 100 points/vertices of the predicted point cloud/mesh. The results demonstrate that, though the point/vertex coordinates are both predicted by fully connected layers, neighboring channels predicts scattered points set but neighboring mesh vertices. This is because the point cloud and chamfer distance are permutation invarant, while the mesh deformation is not. Further, we observe that specific channel generates specific part of the object, e.g. the right leg of the chair.
3. (Extra Credit) Exploring some recent architectures.
3.1 Implicit network (10 points)
### 3.2 Parametric network (10 points)
The network is implemented with five 30*30 atlas.
