Each GIF shows the ground truth (left) next to the optimized result (right), for the first training sample.
All models were trained for 3000 iterations using pretrained ResNet18
features (--load_feat) on --device mps.
Each GIF shows input RGB, ground truth, and prediction.
The same three test examples are used for all three representations.
F1@0.05: 32.305%
F1@0.05: 75.676%
F1@0.05: 63.156%
| Representation | F1@0.05 |
|---|---|
| Point cloud | 75.676% |
| Mesh | 63.156% |
| Voxel grid | 32.305% |
I varied the marching-cubes isovalue for the voxel model. The trained model was kept fixed.
The F1 score changes with the isovalue. A lower value includes more of the predicted occupancy, while a higher value produces a smaller surface.
The best value is a trade-off between keeping the predicted geometry and removing incorrect geometry.
Note: The marching-cubes isovalue is a post-processing parameter, not a training hyperparameter. I used it to measure the sensitivity of the final voxel reconstruction.
I looked at per-example F1 scores and reconstruction errors to see how the models behave on individual examples.
The F1 scores vary across test examples. Some objects are easier to reconstruct than others.
I colored the predicted points based on their distance to the nearest ground-truth point. Larger distances indicate larger reconstruction errors.
The errors are more noticeable around thin parts and parts that are not clearly visible in the input image.
This is especially noticeable for the voxel model because of its limited resolution. The model also has to predict the geometry that is not visible from the input view.