Contents
| Optimized voxel grid (left) and ground truth (right) |
|---|
![]() |
| Optimized point cloud (left) and ground truth (right) |
|---|
![]() |
| Optimized mesh (left) and ground truth (right) |
|---|
![]() |
Setup. I used --load_feat for every model in sections 2 and 3. Instead of 100k iterations, I trained each model for up to 20k and kept the checkpoint with the best F1@0.05 on 10% of the training chairs held out for validation. The test set was only used for the numbers reported here. The decoder architectures are in model.py.
The model checkpoint for this task is checkpoint_vox.pth
My voxel model gets F1@0.05 = 69.0 on the test set.
| Input | Predicted voxels (left) and ground truth (right) |
|---|---|
![]() | ![]() |
![]() | ![]() |
![]() | ![]() |
The model checkpoint for this task is checkpoint_point.pth
My point cloud model gets F1@0.05 = 77.7 on the test set.
| Input | Predicted points (left) and ground truth (right) |
|---|---|
![]() | ![]() |
![]() | ![]() |
![]() | ![]() |
The model checkpoint for this task is checkpoint_mesh.pth
My mesh model gets F1@0.05 = 72.3 on the test set.
| Input | Predicted mesh (left) and ground truth (right) |
|---|---|
![]() | ![]() |
![]() | ![]() |
![]() | ![]() |
| Representation | Test F1@0.05 |
|---|---|
| Point cloud | 77.7 |
| Mesh | 72.3 |
| Voxel grid | 69.0 |
Points do best, then meshes, then voxels. On validation, recall is similar for all three (about 75 to 77), so most of the gap is precision where the mesh and voxel models put surface where there is no chair.
The main hyperparameter I varied is the positive-class weight (pos_weight) in the voxel BCE loss. I also looked at the mesh smoothness weight and the initial mesh. All numbers here are validation F1@0.05.
| Voxel loss (1,024-wide MLP decoder) | Val F1 | Precision | Recall |
|---|---|---|---|
Plain BCE (pos_weight 1) | 57.8 | 69.8 | 53.4 |
pos_weight 1.5 | 66.0 | 71.1 | 64.0 |
pos_weight 2 | 67.3 | 65.8 | 70.4 |
pos_weight 4 | 57.9 | 51.7 | 67.3 |
| Input | Plain BCE, pos_weight 2, ground truth |
|---|---|
![]() | ![]() |
![]() | ![]() |
Only about 5% of the cells are occupied, so with plain BCE the model plays it safe with high precision, low recall, and thin chairs mostly disappear. pos_weight 2 raises F1 by about 9 points, and 4 overshoots and fills in too much.
I wanted to know if pos_weight actually teaches the model something or mostly moves the decision threshold. At the optimum, thresholding a weight-w model at 0.5 is the same as thresholding a plain-BCE model at 1/(1+w). So I re-evaluated the plain-BCE checkpoint at those isovalues:
| Equivalent threshold 1/(1+w) | Plain BCE model at that isovalue | pos_weight w model at 0.5 |
|---|---|---|
| 0.4 (w = 1.5) | 65.7 | 65.8 |
| 0.33 (w = 2) | 66.2 | 67.3 |
| 0.2 (w = 4) | 55.2 | 58.0 |
The two columns are close. The plain model already ranks the cells well, but its probabilities are too low for 0.5 and lowering the isovalue to 1/3 gets back 8.3 of the 9.4 points. This also explains the sweet spot near 2.
w_smooth | Val F1 |
|---|---|
| 0.01 (2.3 model) | 71.0 |
| 0.1 (starter default) | 69.8 |
| 1.0 | 66.3 |
Less smoothing gives higher F1. Laplacian smoothing pulls every vertex towards its neighbours, which makes it difficult to fit the large uneven moves the sphere needs to reach the legs and corners. The stronger weight looks tidier but still isn't a chair-shaped surface.
Initial mesh (w_smooth 0.1) | Val F1 |
|---|---|
| Unit ico-sphere | 69.8 |
| Chair-sized ellipsoid | 69.9 |
| Chair-sized box | 69.7 |
| Sphere, ellipsoid, box, ground truth (chair 300) |
|---|
![]() |
The starting shape makes almost no difference. The decoder predicts every offset from scratch, so shrinking a too-big sphere is learned in the first few thousand steps, and after that all three are limited by a closed surface with no holes.
I used the point cloud model.
Row 2: each predicted point coloured by its distance to the nearest ground-truth point (red hurts precision). Row 3: each ground-truth point coloured by its distance to the nearest predicted point (red hurts recall). Blue is a hit at the 0.05 threshold and red is a miss.
The predicted points are mostly fine (precision 79% to 93%) apart from the halo. But for recall the performance isn't great. On the tall chair 150 the whole upper back is missed (recall 44%), and on the armchair (0) and barrel chair (300) the outer edges are missed. So the model gets the middle of the chair right and misses whatever sticks out.
I blended the image features of the armchair (0) and the tall chair (150) and decoded each blend. The shape changes smoothly from wide and low to narrow and tall, with no odd shapes along the way. The proportions change first, and leg-like strands only show up from about alpha 0.6, so the feature mostly encodes overall chair dimensions instead of structure.
The model checkpoint for this task is checkpoint_implicit.pth.
My implicit model gets F1@0.05 = 58.5 on the test set.
| Input | Implicit prediction (left) and ground truth (right) |
|---|---|
![]() | ![]() |
![]() | ![]() |
![]() | ![]() |
The implicit model is about 10 points below the voxel MLP (58.5 against 69.0). Its outputs are very smooth. A small MLP on raw coordinates fits smooth functions easily and sharp ones slowly, and thin legs are the sharp part. I also tried positional encoding of (x, y, z), which learned faster but ended at the same F1. The voxel MLP has 242M parameters against 0.23M here, and its per-cell output weights are probably where its extra detail comes from.
The model checkpoint for this task is checkpoint_parametric.pth.
My parametric model gets F1@0.05 = 72.6 on the test set.
| Input | Predicted points, patch surfaces, ground truth |
|---|---|
![]() | ![]() |
![]() | ![]() |
![]() | ![]() |
Patch surfaces: the decoder evaluated on a regular 24x24 (u, v) grid per patch and connected into triangles. Each colour is one patch.
It scores 5 points below the point decoder (72.6 against 77.7). I think the reason is continuity because each patch is a smooth map of a square, so it forms sheets and has a hard time following thin, separate legs. The patch surfaces show this with a few large sheets for the back and seat that often overshoot the real chair. The upside is that (u, v) is an input, so I can sample as many points as I want from a model trained with 1,000.
The model checkpoints for this task are checkpoints/point3c.pth (three classes) and checkpoint_point.pth (chairs only). I set use_full_dataset = True in dataset_location.py and retrained the point model from 2.2 with the same settings on split_3c.json (6,100 chairs, 3,640 planes, 3,146 cars). Then I evaluated both models on the same 678 chair test objects.
| Trained on | F1@0.05 on the chair test set |
|---|---|
| Chairs only | 77.5 |
| Chairs, planes and cars | 76.5 |
Both from the same eval run (the chair-only model got 77.7 in its own run in 2.2).
F1 vs threshold, three-class model on the chair test set.
| Test class | Objects | Three-class model F1@0.05 |
|---|---|---|
| Chair | 678 | 76.7 |
| Car | 368 | 93.4 |
| Plane | 405 | 95.7 |
| All | 1,451 | 86.3 |
| Input | Trained on chairs, trained on three classes, ground truth |
|---|---|
![]() | ![]() |
![]() | ![]() |
![]() | ![]() |
| Input | Three-class model: prediction (left) and ground truth (right) |
|---|---|
![]() | ![]() |
![]() | ![]() |
Training on three classes didn't help the chairs in my case since F1 on the chair test set is about 1 point lower and the predicted chairs look almost the same as before. The planes and cars themselves come out as the right kind of object and score above 93 on average, since they are compact shapes with less variety than chairs.
The outputs are also not more diverse. For both models, the predicted chair sizes (width, height and depth over the test set) vary about 30% less than the real ones, so both pull unusual chairs towards an average chair, and the three-class model's predictions are just as consistent from chair to chair as the chair-only model's. I think this is because the encoder is frozen (--load_feat), so the extra classes can only change the decoder, and a decoder that also has to produce planes and cars has less room left for chair detail. It also saw each chair about half as often in the same number of steps.