16-825 Learning for 3D Vision: Assignment 5
16-825, Fall 2026 · Zijin Ye (Andrew ID: zijiny)
Point cloud classification and segmentation with PointNet, robustness analysis, and PointNet++ (bonus).
All results below were produced with the submitted code on a Google Colab A100 GPU.
| Model | Classification test accuracy | Segmentation test accuracy |
|---|---|---|
| PointNet (Q1, Q2) | 97.48% | 90.50% |
| PointNet++ SSG (Q4) | 98.32% | 91.13% |
Models and training setup
Both PointNet models follow the paper without the T-Net transformation blocks. Every "shared MLP" is a
1×1 Conv1d + BatchNorm + ReLU applied to each point independently.
- Classification: shared MLP 3 → 64 → 64 → 64 → 128 → 1024, max-pool over points to a 1024-d global feature, then FC 512 → 256 → 3 (BatchNorm + ReLU, dropout 0.3 before the last layer).
- Segmentation: a per-point local feature (3 → 64 → 64) is concatenated with the 1024-d global feature (64 → 64 → 128 → 1024, max-pooled), and the 1088-d vector of each point goes through 512 → 256 → 128 → 6.
Training uses Adam (learning rate 0.001, cosine decay to 1e-5), batch size 32 and cross-entropy loss. At every training step each object is represented by a fresh random subset of 4,096 of its 10,000 points, which is faster and acts as light augmentation. Testing always uses all 10,000 points. As in the starter code, the checkpoint with the best test accuracy is kept. The classifier trained for 50 epochs (3.8 s/epoch, best at epoch 44) and the segmenter for 60 epochs (4.9 s/epoch, best at epoch 54).
Q1. Classification Model
Test accuracy: 97.48% (929 of 953 test objects; 24 mistakes). Per class: chair 99.84%, vase 91.18%, lamp 94.02%.
| ground truth ↓ / predicted → | chair | vase | lamp |
|---|---|---|---|
| chair | 616 | 0 | 1 |
| vase | 1 | 93 | 8 |
| lamp | 2 | 12 | 220 |
Random test objects
Five test objects drawn at random (seed 0), with the predicted class. Four are chairs because chairs make up 65% of the test set.





Failure cases (at least one per class)





Interpretation. Chairs are almost never wrong: they are the majority class and have a distinctive layout (legs, a flat seat, a back). Most errors are between vases and lamps: 8 vases are called lamps and 12 lamps are called vases. Both classes are mostly round, rotationally symmetric shells, and PointNet decides from a single max-pooled global feature that captures the overall silhouette but not small local details such as a bulb socket, a cord or the opening of a vase. Each failure looks like a typical member of the predicted class:
- #543 chair → lamp: a very tall, thin back slab on a tiny frame has the silhouette of a floor lamp (a tall pole on a small base).
- #620 vase → lamp: an elongated egg shape with a hole in its side, like a lamp shade or lamp body.
- #651 vase → lamp: a bowl with an orchid sticking out; the thin stem with a cluster on top looks like a desk lamp's arm and head.
- #787 lamp → vase: a lamp made of two stacked spheres with no pole, which looks like a round vase.
- #944 lamp → chair: a wide, bell-shaped shade with a flat disc on top. Without a stem there is no lamp cue, and the broad flat surfaces look more like a stool.
Q2. Segmentation Model
Test accuracy: 90.50% (fraction of correctly labelled points over all 617 test chairs). Per-object accuracy: mean 90.50%, median 94.02%, worst 43.44%, best 99.73%.
Colours follow the handout (white, magenta, cyan, yellow, blue, red for parts 0–5), except that part 0 is drawn in gray because white points are invisible on the white background. The part names are my reading of the visualizations; the dataset only provides numbers. The per-part accuracy is the fraction of points of that part that are labelled correctly:
| part (colour) | share of test points | test chairs containing it | PointNet | PointNet++ |
|---|---|---|---|---|
| rare part (0) | 0.33% | 11 | 94.3% | 78.8% |
| headrest (1) | 0.13% | 8 | 15.6% | 12.3% |
| back (2) | 31.97% | 617 | 92.4% | 93.8% |
| armrests (3) | 9.13% | 284 | 76.2% | 82.2% |
| legs / base (4) | 20.31% | 599 | 90.1% | 91.4% |
| seat (5) | 38.14% | 616 | 92.7% | 91.3% |
Good predictions (random test objects)


Chair #50 (random test object): 88.78%


Chair #212 (random test object): 92.07%


Chair #251 (random test object): 92.42%


Chair #498 (random test object): 92.94%
Bad predictions (the two worst test objects)


Chair #235 (one of the two worst test objects): 43.44%


Chair #26 (one of the two worst test objects): 47.60%
Interpretation. On ordinary chairs with clearly separated parts (#50, #212, #251, #498) the prediction is close to the ground truth, and the remaining errors sit on part boundaries: where the seat meets the back or the legs, the label depends on an arbitrary cut. The two worst objects are both sofa-like chairs made of blocky cushions. For #235, the model falls back on a "typical chair by height" layout: back at the top, seat in the middle, legs at the bottom. The real object has almost no legs (1% of its points), and its large side blocks are armrests. Its ground truth is also noisy: armrest and seat points are interleaved. #26 has a headrest cushion (magenta). That part covers only 0.13% of the points and appears in 8 test chairs, so the model almost never predicts it (15.6% of headrest points are labelled correctly). In short, PointNet segments well when the per-point labels can be read off position and the global shape is a standard chair, and it struggles with rare parts and unusual shapes.
Q3. Robustness Analysis — Experiment 1: rotation
Procedure. Every test object is rotated about the x-axis by θ ∈ {0, 15, 30, 45, 60, 90, 135, 180}°.
The y-axis points up, so the object tips forward; at 90° a chair lies on its back and at 180° it is upside down.
Then the Q1 and Q2 checkpoints are evaluated without any retraining. All 10,000 points are used, and for the
segmentation task the ground-truth labels rotate with the points. The θ = 0° row is exactly the Q1 / Q2 result. The same three classification objects (a chair, a vase and a lamp)
and the same chair for segmentation are shown at every angle. The command is
python robustness.py --experiment rotation.
| rotation | PointNet cls | PointNet seg | PointNet++ cls | PointNet++ seg |
|---|---|---|---|---|
| 0 deg about x | 97.5% | 90.5% | 98.3% | 91.1% |
| 15 deg about x | 95.4% | 84.3% | 98.0% | 89.0% |
| 30 deg about x | 83.3% | 72.8% | 85.3% | 80.0% |
| 45 deg about x | 56.2% | 61.2% | 57.8% | 59.7% |
| 60 deg about x | 35.4% | 44.1% | 49.7% | 38.9% |
| 90 deg about x | 23.5% | 27.0% | 63.3% | 18.7% |
| 135 deg about x | 68.2% | 25.6% | 34.1% | 22.5% |
| 180 deg about x | 58.6% | 34.4% | 41.7% | 38.1% |
cls 97.5% · seg 90.5%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
80.0% correctcls 83.3% · seg 72.8%
pred: chair ✓
pred: lamp ✗
pred: vase ✗
ground truth
73.4% correctcls 35.4% · seg 44.1%
pred: lamp ✗
pred: lamp ✗
pred: chair ✗
ground truth
59.9% correctcls 23.5% · seg 27.0%
pred: vase ✗
pred: vase ✓
pred: chair ✗
ground truth
42.8% correctcls 58.6% · seg 34.4%
pred: chair ✓
pred: lamp ✗
pred: lamp ✓
ground truth
39.0% correctInterpretation. PointNet is not rotation invariant. It reads raw xyz coordinates, has no T-Net, and was trained only on upright objects. Small tilts are harmless (15°: 95.4% / 84.3%), but by 45° accuracy has fallen to 56.2% for classification and 61.2% for segmentation. At 90° it drops to 23.5% / 27.0%: only 3.4% of chairs are recognized, because a chair lying on its back is classified as a vase or lamp. Classification partly recovers at 135° and 180° (68.2%, 58.6%). An upside-down chair still has a seat, legs and a back, and lamps (a pole with a shade) stay lamp-like when flipped. Segmentation does not recover (34.4% at 180°) because the model labels points largely by height. In the 180° example it calls the bottom of the object "legs" even though that is now the back, and it calls the old legs "back". The model has learned the canonical orientation of the dataset; rotation augmentation or a learned alignment (T-Net) would be needed to handle rotated inputs.
Q3. Robustness Analysis — Experiment 2: number of points
Procedure. Instead of all 10,000 points, each test object keeps N ∈ {10000, 5000, 2000, 1000, 500, 200, 100,
50, 10} randomly chosen points. The same random indices are used for every object, with a fixed seed. The unchanged
Q1 and Q2 checkpoints are evaluated, and segmentation accuracy is computed over the kept points. The command is
python robustness.py --experiment num_points.
| points per object | PointNet cls | PointNet seg | PointNet++ cls | PointNet++ seg |
|---|---|---|---|---|
| 10000 points | 97.5% | 90.5% | 98.3% | 91.1% |
| 5000 points | 97.4% | 90.6% | 98.2% | 91.2% |
| 2000 points | 97.3% | 90.4% | 97.9% | 91.2% |
| 1000 points | 97.6% | 90.0% | 97.1% | 90.8% |
| 500 points | 97.3% | 88.6% | 91.0% | 88.1% |
| 200 points | 95.9% | 84.8% | 44.8% | 72.4% |
| 100 points | 94.0% | 82.3% | 24.8% | 48.0% |
| 50 points | 80.6% | 79.2% | 24.6% | 30.1% |
| 10 points | 26.1% | 68.8% | 24.6% | 20.8% |
cls 97.5% · seg 90.5%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
95.4% correctcls 97.6% · seg 90.0%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
93.0% correctcls 95.9% · seg 84.8%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
89.5% correctcls 80.6% · seg 79.2%
pred: chair ✓
pred: lamp ✗
pred: lamp ✓
ground truth
98.0% correctcls 26.1% · seg 68.8%
pred: lamp ✗
pred: lamp ✗
pred: lamp ✓
ground truth
50.0% correctInterpretation. PointNet is very robust to sparse inputs. With only 500 points (5% of the input) it still reaches 97.3% / 88.6%, and with 100 points it reaches 94.0% / 82.3%. The reason is max-pooling: the global feature is determined by a small set of "critical" points that span the shape, and a uniform random subset still covers the same outline. Training on random 4,096-point subsets also exposed the model to varying density. Accuracy only collapses when the outline itself disappears. Vases go first (66% correct at 100 points, 34% at 50), and at 10 points almost everything is classified as a lamp (99.6% of lamps but 2.6% of chairs), since ten scattered points look like a thin, sparse object. Segmentation degrades more gracefully (68.9% at 10 points), because a point's label can still be guessed from its height alone.
Extra experiment: Gaussian noise
Procedure. Independent Gaussian noise N(0, σ²) is added to every coordinate of every test point, for
σ ∈ {0, 0.01, 0.02, 0.05, 0.1, 0.2}. The objects are normalized to a radius of 1, so σ = 0.1 is 10% of the object
radius. The command is python robustness.py --experiment noise.
| noise | PointNet cls | PointNet seg |
|---|---|---|
| sigma 0 | 97.5% | 90.5% |
| sigma 0.01 | 96.7% | 90.2% |
| sigma 0.02 | 96.2% | 89.3% |
| sigma 0.05 | 90.8% | 83.4% |
| sigma 0.1 | 86.9% | 64.4% |
| sigma 0.2 | 73.3% | 44.5% |
cls 97.5% · seg 90.5%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
80.0% correctcls 90.8% · seg 83.4%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
68.2% correctcls 86.9% · seg 64.4%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
ground truth
55.1% correctcls 73.3% · seg 44.5%
pred: chair ✓
pred: chair ✗
pred: lamp ✓
ground truth
45.2% correctInterpretation. Light jitter (σ ≤ 0.02) barely matters. Classification degrades slowly (86.9% at σ = 0.1) because the overall box-like shape of a chair survives heavy noise: chairs stay above 99% even at σ = 0.2. Lamps fail first (64% at σ = 0.05, 27% at σ = 0.2) because their thin poles and cords dissolve into the noise. Segmentation is much more sensitive (64.4% at σ = 0.1, 44.5% at σ = 0.2): part boundaries blur, and at σ = 0.2 the model labels most of the noisy cloud as seat.
Q4. Bonus — Locality: PointNet++
Model. I implemented PointNet++ with single-scale grouping (SSG) in plain PyTorch, with no custom CUDA kernels. PointNet applies one MLP to every point and then pools over the whole object, so it never looks at neighbourhoods. PointNet++ builds features hierarchically from local regions instead. Each set abstraction (SA) layer works in three steps:
- Farthest point sampling picks a set of well-spread centroids.
- A ball query collects up to K neighbours within radius r of each centroid. The neighbours' coordinates are expressed relative to the centroid, so the features describe local shape rather than absolute position.
- A shared mini-PointNet (MLP + max-pool) turns each neighbourhood into one feature vector.
- Classification: SA(512 centroids, r = 0.2, K = 32, MLP 64-64-128) → SA(128, r = 0.4, K = 64, MLP 128-128-256) → global SA (MLP 256-512-1024 + max-pool) → FC 512 → 256 → 3 with dropout 0.4.
- Segmentation: the same encoder followed by three feature propagation layers. Each one interpolates features back to the denser point set (inverse-distance weighting over the 3 nearest neighbours), concatenates the skip features from the matching encoder level and applies a shared MLP: [256, 256], [256, 128], then [128, 128, 128]. A per-point head 128 → 6 with dropout 0.5 follows.
Training uses the same recipe as Q1 and Q2 (Adam, cosine decay, batch 32), with 40 epochs, random 2,048-point subsets per object and mixed precision. That takes 26 s/epoch for classification and 19 s/epoch for segmentation on an A100. Testing again uses all 10,000 points.
Test accuracy compared with Q1 and Q2
| Task | PointNet (Q1 / Q2) | PointNet++ | Change |
|---|---|---|---|
| Classification (overall) | 97.48% (24 errors) | 98.32% (16 errors) | +0.84 points, one third fewer errors |
| chair / vase / lamp | 99.84% / 91.18% / 94.02% | 99.68% / 93.14% / 97.01% | vase and lamp improve |
| Segmentation (all points) | 90.50% | 91.13% | +0.62 points |
| median / worst chair | 94.02% / 43.44% | 94.74% / 39.68% |
Farthest point sampling starts from a random point, so PointNet++ test accuracy varies by about one object between runs. The evaluation script gives 98.32% / 91.13%, and the end-of-epoch evaluation during training gave 98.43% / 91.27%.
| ground truth ↓ / predicted → | chair | vase | lamp |
|---|---|---|---|
| chair | 615 | 0 | 2 |
| vase | 0 | 95 | 7 |
| lamp | 0 | 7 | 227 |
Per-part accuracies for both models are in the Q2 table. PointNet++ improves the back (92.4% → 93.8%), armrests (76.2% → 82.2%) and legs (90.1% → 91.4%), the parts whose boundaries are defined by local geometry. It is slightly worse on the seat (92.7% → 91.3%) and clearly worse on the very rare part 0 (94.3% → 78.8%).
Classification: the Q1 failure cases
PointNet++ classifies 3 of the 5 Q1 failures correctly. The orchid bowl (#651) and the egg-shaped vase (#620) are now vases, and the bell-shaped lamp (#944) is now a lamp. Local features capture the vase opening and the shape of the thin stem, details the single global PointNet feature tends to wash out. The tall chair (#543) and the stacked-sphere lamp (#787) fool both models.





The remaining PointNet++ errors are still mostly between vases and lamps: ornate urns with a pedestal (#631, #650) and lamps with a vase-like body (#793, #859). PointNet classifies all five of these correctly, so the two models make different mistakes.





Segmentation: the Q2 objects side by side
The same chairs as in Q2, showing the ground truth, the PointNet prediction and the PointNet++ prediction.



Chair #50: PointNet 88.78% → PointNet++ 90.22%



Chair #212: PointNet 92.07% → PointNet++ 90.10%



Chair #251: PointNet 92.42% → PointNet++ 94.09%



Chair #235: PointNet 43.44% → PointNet++ 51.44%



Chair #26: PointNet 47.60% → PointNet++ 57.41%
PointNet++ helps most on the hard sofa-like chairs: #235 goes from 43.4% to 51.4% and #26 from 47.6% to 57.4%. On #235 it labels the side blocks as armrests instead of applying PointNet's height-based back / seat / legs layout, and on #26 it finds the armrest and the seat cushions. Neither model finds the headrest of #26. On ordinary chairs the two models are within about two points of each other. The two worst PointNet++ chairs are shown below. On #426 the boxy skirt under the seat is labelled as the rare part 0 (gray), and on #238 the headrest and the base are confused.


Chair #238: PointNet++ 39.68% (PointNet got 65.85% on this chair)


Chair #426: PointNet++ 46.11% (PointNet got 78.62% on this chair)
Robustness compared with PointNet
The orange curves in the rotation and number-of-points plots repeat the Q3 experiments for PointNet++, and the tables there list both models.
- Rotation: PointNet++ is slightly more robust to small tilts (15°: 98.0% / 89.0% against 95.4% / 84.3%). Beyond 30° both models fail. PointNet++ classifies better at 60–90° and worse at 135–180°, and its segmentation is no better at any large angle. The local coordinates are relative to each centroid but still expressed in the global frame, so locality alone does not give rotation invariance.
- Number of points: PointNet++ is much less robust to sparse inputs. Classification falls to 91.0% at 500 points and 44.8% at 200. From 100 points down, almost every object is predicted as a lamp (24.6% = the share of lamps). The ball queries use fixed radii (0.2 and 0.4), so with few points each ball holds only one or two neighbours, and the local features look nothing like the dense 2,048-point neighbourhoods seen in training. The PointNet++ paper addresses this with multi-scale grouping and random input dropout during training, which I did not use. The GIFs below show the effect for the same objects as in Q3.
cls 97.1% · seg 90.8%
pred: chair ✓
pred: vase ✓
pred: lamp ✓
94.6% correctcls 44.8% · seg 72.4%
pred: lamp ✗
pred: lamp ✗
pred: lamp ✓
78.5% correctcls 24.6% · seg 30.1%
pred: lamp ✗
pred: lamp ✗
pred: lamp ✓
44.0% correct