srishtig
Optimized (left), target (right)
Optimized (left), target (right)
Optimized (left), target (right)
Input, prediction, ground truth


F1@0.05 = 77.0
Input, prediction, ground truth


F1@0.05 = 87.0
Input, prediction, ground truth


F1@0.05 = 77.4
F1@0.05: point cloud 87.0, mesh 77.4, voxel 77.0.
Point clouds do best because each point can go anywhere, so the model can place points on thin parts like legs, and the chamfer loss is close to what F1 measures. The mesh is a deformed sphere, so it can't make holes and ends up stretching surfaces across gaps between legs, but on flat parts like the seat it is very accurate (best F1 at 0.01). Voxels are limited by the 32x32x32 grid, which is too coarse for thin legs and slats.
I trained the point cloud model with 1000 and 5000 points.
| n_points | F1@0.01 | F1@0.02 | F1@0.03 | F1@0.04 | F1@0.05 |
|---|---|---|---|---|---|
| 1000 | 3.7 | 21.2 | 47.6 | 67.9 | 79.9 |
| 5000 | 6.9 | 34.9 | 63.1 | 78.9 | 87.0 |
1000 (left), 5000 (right)
5000 points is better at every threshold, most at 0.02 to 0.03. With 1000 points the points are about 0.015 apart on the surface, which is larger than the small thresholds, so even a perfect prediction would score low there (about 27 at 0.01 and 72 at 0.02). With 5000 points the spacing drops to about 0.007, so the model can actually cover the surface finely enough.
To see where the models fail, I colour each ground truth point by its distance to the nearest predicted point (red = missed), and each predicted point by its distance to the ground truth (red = wrong). Best 3 and worst 3 test examples for the point cloud model:
I also measured how much of the ground truth surface is missed at each height of the chair, over the whole test set (voxel, point, mesh):
All models miss the legs the most, then the top of the backrest. The seat is the most reliable. The best examples are normal four-legged chairs, and the worst are unusual ones (a bar stool, odd viewpoints), where the model just predicts a generic chair.
The network takes random 2D points in [0,1]^2, concatenates each with the image feature, and maps them to 3D points with an MLP. I used 5 patches (one MLP each) with 1000 points each, trained with chamfer loss.
Input, prediction, ground truth


F1@0.05 = 82.8
It does better than voxels and meshes but worse than the plain point decoder. Since it is a function of the 2D input, it can output any number of points at test time.