contents
setup
both pointnets are the basic version without the transform blocks. cls: shared mlp 3→64→64→64→128→1024 on every point, max pool, then fc 512→256→3. seg: glue each point's 64-d feature to the 1024-d global feature and run a per-point mlp down to 6 classes. adam, batch 32, cross entropy, trained on one L40S.
| PointNet cls | PointNet seg | PointNet++ cls | PointNet++ seg | |
|---|---|---|---|---|
| params | 0.81M | 0.88M | 1.46M | 1.40M |
| epochs | 150 | 150 | 100 | 100 |
| train points / object | all 10k | all 10k | 2048 | 4096 |
| lr | 1e-3 | 1e-3 | 1e-3, cosine | 1e-3, cosine |
| train time | 36 min | 45 min | 28 min | 30 min |
| best test acc (epoch) | 97.90% (125) | 89.95% (132) | 98.32% (68) | 91.64% (51) |
| last epoch test acc | 96.85% | 87.81% | 98.22% | 91.54% |
best_model.pt is the epoch with the best test acc (that's how the starter picks it), so the numbers below are a little optimistic. the last-epoch row shows by how much.
Q1. classification
| class | acc | correct |
|---|---|---|
| chair | 100.00% | 617 / 617 |
| vase | 88.24% | 90 / 102 |
| lamp | 96.58% | 226 / 234 |
| gt \ pred | chair | vase | lamp |
|---|---|---|---|
| chair | 617 | 0 | 0 |
| vase | 0 | 90 | 12 |
| lamp | 2 | 6 | 226 |

random test objects
6 random ones, colored by predicted class (chair vase lamp). all correct.






failures
most confident wrong prediction for each class. no chair was wrong, so i show the closest call, plus a real chair failure from the 15° tilt test in Q3 (no new runs, just the existing results).




- all 20 mistakes are vase/lamp mixups (12 vases called lamps, 6 lamps called vases) plus 2 lamps called chairs. zero chair mistakes.
- vase #651 is a potted orchid: tall curvy stem with flowers at the top. honestly it looks like an arc floor lamp. pointnet only gets one max-pooled global feature, so it sees "thin stem + blob on top + base" and says lamp.
- lamp #922 is a candelabra with thin arms coming off a base, which looks like branches in a vase.
- chairs are 2/3 of the training data and super distinctive, so none fail. the closest call (#406, p = 0.89) is a barrel chair that looks kind of like a container. tilting the test set by just 15° makes 10 chairs fail, all as lamps. #543 is a really tall ladder-back chair, and tilted it's basically a pole on a small base, so lamp.
- vase is the smallest class (741 train vs 1554 lamps and 4489 chairs), so anything ambiguous leans lamp. that's why vase has the lowest acc (88.2%).
Q2. segmentation
per chair: mean 90.0%, median 93.2%, worst 35.4%, 29 chairs under 70%. part names are my guess from where they sit on the chair.
| part | % of points | acc | IoU |
|---|---|---|---|
| 0: rare part 0 | 0.3% | 90.3% | 0.899 |
| 1: rare part 1 (headrest?) | 0.1% | 13.4% | 0.097 |
| 2: back | 32.0% | 91.5% | 0.860 |
| 3: arms | 9.1% | 77.7% | 0.655 |
| 4: legs / base | 20.3% | 89.0% | 0.823 |
| 5: seat | 38.1% | 92.4% | 0.824 |

5 chairs: the 2 worst, then median, 75th percentile and best





- normal chairs are fine (93.2%, 96.5%, 99.6%). the errors are mostly on part boundaries like seat/legs and seat/back. each point only sees its own xyz plus the global feature, no neighbors, so it's pretty much labeling by height and position.
- #26 (35.4%) is a blocky sofa chair with no legs. it calls the bottom of the base legs (low points are usually legs) and merges the arms into the seat.
- #351 (46.4%) is a barrel chair where the sides wrap around and are labeled arms. it splits that shell by height into back and legs instead. the gt is kind of ambiguous there anyway.
- the two rare parts (0.3% and 0.1% of points) are what pull the mIoU down.
Q3. robustness
for all of these i used the trained q1/q2 models and only messed with the test inputs, no retraining. full
test sets, run with python robustness.py (same as eval_cls.py /
eval_seg.py with --rot, --num_points, --noise). the first
column of each table is the q1/q2 result. the PointNet++ rows are for q4. red = more than 10 points below the
unperturbed number. in the gifs each column is one setting, cls is colored by predicted class, seg has gt on
top and predictions below.
experiment 1: rotation
rotate every test cloud about x (y is up), 0 to 180°, so things tip over until they're upside down.

| 0° | 15° | 30° | 45° | 60° | 90° | 120° | 180° | |
|---|---|---|---|---|---|---|---|---|
| cls PointNet | 97.9% | 94.8% | 68.6% | 39.3% | 30.6% | 33.8% | 61.4% | 78.8% |
| cls PointNet++ | 98.3% | 94.4% | 54.4% | 36.9% | 37.9% | 40.5% | 30.7% | 36.4% |
| seg PointNet | 90.0% | 85.1% | 74.0% | 60.7% | 42.5% | 23.8% | 23.3% | 35.6% |
| seg PointNet++ | 91.6% | 89.8% | 79.6% | 63.3% | 42.6% | 23.2% | 21.1% | 37.3% |






- no t-net and only upright training data, so it drops fast. cls goes 97.9% → 94.8% at 15° → 39.3% at 45°. seg goes 90.0% → 85.1% → 60.7%.
- tilted chairs mostly turn into lamps (chair acc 24% at 45°) while lamps stay at 89%. lamp is kind of the default for weird tilted shapes.
- cls comes back a bit at 180° (78.8%) because an upside-down chair still has a seat, back and straight legs.
- seg never comes back (23.8% at 90°, 35.6% at 180°) since the labels are tied to height. that's worse than calling every point seat (38%).
experiment 2: number of points
keep a random N of the 10k points per object (same indices for every object), N from 10000 down to 10.

| 10,000 | 5,000 | 2,000 | 1,000 | 500 | 100 | 50 | 10 | |
|---|---|---|---|---|---|---|---|---|
| cls PointNet | 97.9% | 98.0% | 97.5% | 97.2% | 97.5% | 92.5% | 79.0% | 28.8% |
| cls PointNet++ | 98.3% | 98.3% | 98.2% | 98.1% | 91.5% | 24.7% | 24.6% | 24.6% |
| seg PointNet | 90.0% | 89.9% | 89.7% | 89.1% | 87.5% | 78.8% | 73.9% | 64.5% |
| seg PointNet++ | 91.6% | 91.5% | 91.5% | 89.7% | 83.5% | 31.5% | 21.6% | 19.9% |






- barely changes until it gets really sparse: cls 97.5% at 500 points and 92.5% at 100, seg 87.5% and 78.8%. max pooling only needs the points near the extremes of each feature (the "critical points") and a random subset still has most of them.
- vases go first (51% at 100 points, 16% at 50) since thin rims and stems disappear. at 10 points almost everything is lamp (28.8%).
- seg holds up better (64.5% even at 10 points) because each label mostly depends on the point's own height.
experiment 3 (extra): noise
add gaussian noise with std σ to every coordinate. objects have radius 1, so σ = 0.1 is 10% of the object.

| 0 | 0.01 | 0.02 | 0.05 | 0.1 | |
|---|---|---|---|---|---|
| cls PointNet | 97.9% | 97.6% | 97.0% | 92.2% | 90.0% |
| cls PointNet++ | 98.3% | 98.1% | 97.1% | 66.3% | 10.8% |
| seg PointNet | 90.0% | 89.7% | 88.8% | 82.6% | 69.0% |
| seg PointNet++ | 91.6% | 91.1% | 89.4% | 81.4% | 73.8% |





pretty robust. cls only goes 97.9% → 90.0% at σ = 0.1, lamps get hit hardest (73%) since their thin parts blur. seg drops more (69.0%) because noise pushes points across the height-based part boundaries.
Q4. bonus: PointNet++
i implemented single-scale PointNet++ in plain pytorch (pointnet2.py). each set abstraction
layer picks centers with farthest point sampling, grabs up to k neighbors within radius r of each center (ball
query), runs a mini pointnet on each neighborhood (coordinates relative to the center) and max pools it. so
features describe local neighborhoods instead of single points, which is the locality pointnet is missing.
- cls: SA(512 centers, r=0.2, k=32, mlp 64-64-128) → SA(128, r=0.4, k=64, 128-128-256) → SA(all, 256-512-1024) → fc 512-256-3
- seg: same encoder (k=64 in the first SA), then 3 feature propagation layers (inverse-distance 3-nn interpolation + skip connection + mlp) back up to all 10k points
- the objects have radius 1, which is what the paper's radii assume. trained on 2048 (cls) / 4096 (seg) random points per object for 100 epochs with cosine lr, tested on all 10k points like q1/q2.
| task | model | params | test acc | details |
|---|---|---|---|---|
| cls | PointNet (q1) | 0.81M | 97.90% | chair 100.0%, vase 88.2%, lamp 96.6% |
| cls | PointNet++ | 1.46M | 98.32% | chair 99.8%, vase 90.2%, lamp 97.9% |
| seg | PointNet (q2) | 0.88M | 89.95% | mIoU 0.693 |
| seg | PointNet++ | 1.40M | 91.64% | mIoU 0.672 |


cls, same objects as q1
left PointNet, right PointNet++, colored by predicted class.








seg, same chairs as q2





robustness
see the PointNet++ rows in the q3 tables and plots.
- locality helps a little for cls (97.90% → 98.32%; 12 objects only PointNet++ gets right vs 8 only PointNet) and more for seg (89.95% → 91.64%, better on 470 of 617 chairs), mostly at part boundaries. the worst q2 chair (#26) goes 35.4% → 64.7%, since neighborhood features can tell the base is one solid block and not legs. it also fixes both q1 failures (#651 and #922).
- mIoU actually drops a bit (0.693 → 0.672). the rare parts are smaller than a ball neighborhood so they get swallowed by the part around them.
- it's way less robust to sparse or noisy input. at 100 points cls is 24.7% (everything becomes lamp) vs 92.5% for PointNet, and seg is 31.5% vs 78.8%. at noise 0.1 cls is 10.8% (everything becomes vase) vs 90.0%. the balls have a fixed radius of 0.2, so sparse clouds leave them almost empty, and noise of 0.1 is half the radius which wrecks the local shape. PointNet's one global max pool doesn't care as much. the paper fixes this by randomly dropping input points during training, i didn't do that here.
- rotation: neither model is invariant. PointNet++ is a bit better at small angles for seg (79.6% vs 74.0% at 30°) but worse for cls at big angles (36.4% vs 78.8% at 180°).