Shruti Jain · spjain
Test accuracy: 95.07% on 953 objects, using 2,048 points per object.
| true chair, pred lamp | true vase, pred lamp | true lamp, pred vase |
|---|---|---|
![]() |
![]() |
![]() |
| The back of the chair forms a semicircle on the top which the model confuses for the lamp shade and hence gets it wrong | the long neck of the vase is confusing for the model and it thinks its a lamp, the flower on top also looks like a shade | It is likely that in the dataset most vases have thicker round body, so the model thinks this lamp is a vase. The model should ideally also pay attention to the shade of the lamp and deduce its not a vase, but it doesnt happen |
The test point accuracy is 90.33% across 617 chairs, using 2,048 points per chair. Ground truth is on the left and prediction on the right in each pair. Objects 26 and 272 are the two lowest-accuracy predictions.
Object 0 — 94.82% accuracy (1,942/2,048 points). The back, seat and supporting structure are mostly separated correctly, with small errors near their boundaries. The two tall backrest uprights retain their green labels.
Object 26 — 43.26% accuracy (886/2,048 points), bad prediction. The red upper section becomes green, much of the yellow region becomes magenta, and the lower magenta region becomes cyan. These broad region errors affect large parts of the chair rather than just their boundaries.
Object 272 — 46.58% accuracy (954/2,048 points), bad prediction. The green back is preserved, but the magenta seat label spreads downward across most of the cyan base. This large base-to-seat confusion explains the low point accuracy.
Object 71 — 93.02% accuracy (1,905/2,048 points). Most of the back, arms, seat and base match well. The small red region at the top is merged into the green back.
Object 131 — 95.75% accuracy (1,961/2,048 points). The back, seat and legs are separated cleanly, with small differences around the seat/support boundary. The narrow supports remain mostly cyan, while the rounded back stays green.
Both experiments keep the Q1/Q2 model weights fixed and evaluate every test object with sampling seed 42. Each example follows the same object across all tested settings, starting with the 2,048-point, unrotated baseline.
Rotate the raw coordinates by each angle about the origin’s y-axis, keeping the same sampled points and corresponding segmentation labels.
| Rotation | Classification accuracy | Segmentation point accuracy |
|---|---|---|
| 0° (baseline) | 95.07% | 90.33% |
| 30° | 86.46% | 80.13% |
| 60° | 72.09% | 65.19% |
| 90° | 66.53% | 55.63% |
| 180° | 29.17% | 60.75% |
Chair 417 from Q1
Lamp 837
Each column uses the same angle for ground truth (top) and prediction (bottom). Prediction accuracy is shown for every angle.
Chair 0 from Q2
Chair 26 from Q2
At 180°, classification drops by 65.90 percentage points and segmentation by 29.58 points relative to the baseline. Chair 417 remains correctly classified through 90° but becomes a vase at 180°. The asymmetric lamp 837 is correct at 0° and 30°, becomes a vase at 60° and 90°, and a chair at 180°. Chair 0’s segmentation worsens at 60° and 90° before partly recovering at 180°; chair 26 is difficult at every angle and becomes almost entirely magenta at 180°. These changes show that the models depend on input orientation, with a different response for each shape.
Keep coordinates unrotated and take progressively shorter prefixes of the same per-object point permutation, producing nested subsets of 2,048, 1,024, 512 and 128 points. Segmentation accuracy is measured only on the retained points, with their matching ground-truth labels.
| Points per object | Classification accuracy | Segmentation point accuracy |
|---|---|---|
| 2,048 (baseline) | 95.07% | 90.33% |
| 1,024 | 95.07% | 90.13% |
| 512 | 94.65% | 89.39% |
| 128 | 93.28% | 84.22% |
Chair 417 from Q1
Lamp 837
Each column shows ground truth (top) and prediction (bottom) on exactly the same retained points, with the corresponding prediction accuracy.
Chair 0 from Q2
Chair 26 from Q2
Reducing to 128 points costs 1.78 percentage points in classification and 6.11 points in segmentation. Both classification examples stay correct at every point count, retaining enough overall shape even as thin structures become sparse. Chair 26 remains difficult and falls to 35.16% at 128 points, while chair 0’s sampled-point score rises from 94.82% to 97.66%. That increase compares different evaluated subsets and does not establish better full-cloud segmentation; the full test-set score falls as points are removed.
The PointNet++ variant uses farthest-point sampling to 512 then 128 centers, 32 nearest neighbors per center, and shared MLPs on relative coordinates followed by local max pooling. Classification uses global max pooling and an MLP head. Segmentation propagates features through three-neighbor interpolation weighted by inverse squared distance, combined with skip features to produce per-point predictions.
Each model uses its best validation checkpoint. Both models are evaluated on identical raw 2,048-point test inputs sampled with seed 42.
| Task | PointNet baseline | PointNet++ variant |
|---|---|---|
| Classification accuracy | 95.07% | 96.75% |
| Segmentation point accuracy | 90.33% | 91.58% |
Locality improves classification by 1.68 percentage points and segmentation by 1.25 points. It corrects both displayed Q1 classification failures and recovers the red upper region and more of the yellow region on the difficult Q2 chair 26.
Chair 406: baseline predicts lamp; the local model predicts chair.
Vase 619: baseline predicts lamp; the local model predicts vase.
Chair 0: 94.82% → 95.51% point accuracy. Panels show ground truth, PointNet and the PointNet++ variant.
Chair 26: 43.26% → 75.20% point accuracy.