Test accuracy: 97.69% (931 / 953). Per class: chair 100%, vase 91.2%, lamp 94.4%.
| true \ predicted | chair | vase | lamp |
|---|---|---|---|
| chair | 617 | 0 | 0 |
| vase | 0 | 93 | 9 |
| lamp | 1 | 12 | 221 |
![]() #312, true chair, predicted chair |
![]() #160, true chair, predicted chair |
![]() #629, true vase, predicted vase |
![]() #640, true vase, predicted vase |
![]() #760, true lamp, predicted lamp |
![]() #864, true lamp, predicted chair (wrong) |
No test chair was misclassified, so the chair column shows the only object predicted as chair and the least confident chair.
| Chair | Vase | Lamp |
|---|---|---|
![]() #864: true lamp, predicted chair |
![]() #618: true vase, predicted lamp |
![]() #721: true lamp, predicted vase |
![]() #406: true chair, predicted chair, but only 46% chair vs 45% lamp |
![]() #651: true vase, predicted lamp |
![]() #726: true lamp, predicted vase |
Interpretation: Errors are mostly vase vs. lamp, which share shapes: cylinders (#618, #721), round bulbous bodies (#726), and a thin arching stem resembling a lamp arm (#651). Chairs, the majority class with a distinctive structure, are never missed. Wall lamp #864 has no typical lamp silhouette.
Test accuracy: 90.21% (median per-object 93.99%).
Part colors (names inferred from the labels): ■ back, ■ seat, ■ legs/base, ■ arms, ■ headrest, ■ other (e.g. footrest).
| Object | Ground truth | Prediction (accuracy) |
|---|---|---|
| #191 | ![]() | 97.13% |
| #389 | ![]() | 95.55% |
| #581 | ![]() | 87.82% |
| #256 | ![]() | 86.32% |
| #26 (bad) | ![]() | 48.53% |
| #605 (bad) | ![]() | 43.11% |
Interpretation: Typical chairs are segmented almost perfectly, with errors at part boundaries. Failures are unusual shapes or ambiguous labels: the sofa with ottoman (#26), solid side panels labeled as legs but predicted as arms (#605), and a stool top labeled as back (#581).
Q1/Q2 models, no retraining.
Procedure: each test object is tilted about the x axis by a fixed angle before evaluation.
| Rotation | Classification acc. | Segmentation acc. |
|---|---|---|
| 0° (Q1/Q2) | 97.69% | 90.21% |
| 15° | 94.86% | 83.28% |
| 30° | 86.46% | 69.34% |
| 45° | 58.45% | 60.12% |
| 60° | 36.94% | 52.26% |
| 90° | 32.32% | 24.89% |
| 180° | 75.13% | 35.08% |
| 30° | 60° | 90° | |
|---|---|---|---|
| Classification, chair #312 (correct at 0°) | ![]() predicted chair |
![]() predicted lamp |
![]() predicted vase |
| Segmentation, chair #191: prediction (97.13% at 0°) | ![]() 69.92% |
![]() 48.54% |
![]() 27.89% |
| Ground truth | ![]() |
![]() |
![]() |
Interpretation: Without transformation blocks or rotation augmentation, PointNet is not rotation invariant and degrades beyond ~15°. Segmentation relies heavily on height, so tilted parts are relabeled. Upside-down (180°) classification partly recovers since an inverted chair still looks chair-like.
Procedure: evaluate on n randomly sampled points per object (--num_points).
| Points per object | Classification acc. | Segmentation acc. |
|---|---|---|
| 10000 (Q1/Q2) | 97.69% | 90.21% |
| 5000 | 97.69% | 90.35% |
| 1000 | 97.38% | 90.38% |
| 500 | 96.64% | 89.54% |
| 100 | 93.18% | 81.97% |
| 50 | 80.38% | 77.73% |
| 10 | 27.07% | 51.75% |
| 1000 points | 100 points | 50 points | |
|---|---|---|---|
| Classification, chair #312 | ![]() predicted chair |
![]() predicted chair |
![]() predicted lamp |
| Segmentation, chair #191: prediction | ![]() 97.20% |
![]() 85.00% |
![]() 84.00% |
| Ground truth | ![]() |
![]() |
![]() |
Interpretation: Max-pooling makes PointNet robust to sparsity; accuracy is nearly unchanged down to 500 points and drops only when whole parts lose their points (<100).
PointNet++ (single-scale grouping): set-abstraction layers sample centers by farthest point sampling, group neighbors within a radius, and apply a shared MLP + max-pool to local coordinates, building features hierarchically. For segmentation, feature propagation layers upsample to every input point (3-NN inverse-distance weights) and merge the matching encoder features.
| Task | PointNet (Q1/Q2) | PointNet++ |
|---|---|---|
| Classification | 97.69% | 98.01% |
| Segmentation | 90.21% | 91.18% |
![]() #864, true lamp PointNet: chair PointNet++: lamp |
![]() #618, true vase PointNet: lamp PointNet++: vase |
![]() #651, true vase PointNet: lamp PointNet++: lamp |
![]() #721, true lamp PointNet: vase PointNet++: lamp |
![]() #726, true lamp PointNet: vase PointNet++: lamp |
![]() #629, true vase PointNet: vase PointNet++: vase |
| Object | Ground truth | PointNet (Q2) | PointNet++ |
|---|---|---|---|
| #191 | ![]() | ![]() 97.13% | ![]() 98.43% |
| #581 | ![]() | ![]() 87.82% | ![]() 95.16% |
| #389 | ![]() | ![]() 95.55% | ![]() 88.86% |
| #26 | ![]() | ![]() 48.53% | ![]() 59.68% |
| #605 | ![]() | ![]() 43.11% | ![]() 88.58% |
| Rotation | 0° | 15° | 30° | 45° | 60° | 90° | 180° |
|---|---|---|---|---|---|---|---|
| Cls, PointNet | 97.7 | 94.9 | 86.5 | 58.4 | 36.9 | 32.3 | 75.1 |
| Cls, PointNet++ | 97.9 | 96.7 | 90.2 | 64.8 | 42.6 | 39.0 | 32.5 |
| Seg, PointNet | 90.2 | 83.3 | 69.3 | 60.1 | 52.3 | 24.9 | 35.1 |
| Seg, PointNet++ | 91.2 | 89.5 | 74.9 | 60.5 | 33.6 | 20.0 | 37.2 |
| Points | 10000 | 5000 | 1000 | 500 | 100 | 50 | 10 |
|---|---|---|---|---|---|---|---|
| Cls, PointNet | 97.7 | 97.7 | 97.4 | 96.6 | 93.2 | 80.4 | 27.1 |
| Cls, PointNet++ | 98.1 | 98.2 | 97.8 | 91.0 | 24.6 | 24.6 | 24.6 |
| Seg, PointNet | 90.2 | 90.4 | 90.4 | 89.5 | 82.0 | 77.7 | 51.8 |
| Seg, PointNet++ | 91.3 | 91.4 | 89.7 | 80.8 | 64.7 | 61.2 | 53.2 |