1. Classification Model 30 pts
Implement a PointNet classifier for chairs, vases and lamps. Report the test accuracy, visualize a few random test point clouds with their predicted classes, and visualize at least one failure for each class with an interpretation.
Model and training
Vanilla PointNet without the transformation blocks. A shared per-point MLP (3 → 64 → 64 → 64 → 128 → 1024, each layer Linear + BatchNorm + ReLU) encodes every point independently; a max-pool over the points gives a 1024-d global feature that is invariant to point order; a head (1024 → 512 → 256 → 3, BatchNorm + ReLU + dropout 0.3 on the hidden layers) outputs class logits. Trained with cross-entropy, Adam (lr 1e-3), batch size 32, all 10,000 points per object, for 75 epochs (~20 s/epoch on a 16 GB laptop GPU). The best checkpoint is from the last epoch, 74.
python train.py --task cls --num_epochs 75
python q1_cls_viz.py
Test accuracy: 97.69% (931 / 953)
| Class | Test objects | Accuracy |
|---|---|---|
| Chair | 617 | 99.68% |
| Vase | 102 | 87.25% |
| Lamp | 234 | 97.01% |
| GT ↓ / Pred → | Chair | Vase | Lamp |
|---|---|---|---|
| Chair | 615 | 0 | 2 |
| Vase | 1 | 89 | 12 |
| Lamp | 0 | 7 | 227 |
Almost all errors are vase ↔ lamp confusions. Vases are the smallest class (11% of the test set, and similarly rare in training), so they have the lowest accuracy.
Random test samples
Two random objects per class (sampling uniformly would give mostly chairs, which are 65% of the test set). Each cloud is colored by its predicted class: chair, vase, lamp. All six are classified correctly.






Numbers in parentheses are softmax confidences. Chairs and lamps are classified with near certainty; vases, even when correct, get noticeably lower confidence.
Failure cases
The two most confident mistakes for each ground-truth class (chairs only have two errors in total). Colors again show the predicted class.






Interpretation
- Chairs → lamp. Both failures are atypical chairs. #543 is a very tall, narrow ladder-back chair whose back is several times taller than the seat, a tall thin silhouette that is typical of floor lamps. #406 is extremely flat (thin when viewed from the side), with no clear seat depth, so the usual seat/back/leg structure is weak. The model only got #543 wrong by a small margin (0.51).
- Vases → lamp. Many "vases" in this dataset contain plants. In #651 the orchid's curved stem and flower cluster look like a lamp's arm and shade over a base, and #673 is a tall, thin tiered plant on a small pot, again a lamp-like shape. A global max-pooled feature captures overall shape (tall, thin, something heavy on top or bottom) more than fine surface detail, so these get confused with high confidence.
- Lamps → vase. Both are lamps with a large closed body: #787 is a round glass-globe wall lamp, and #859 is a boxy lantern. Without a visible pole, arm or open shade, they look like a rounded closed container, which is what most vases are.
- Overall, the classes overlap in shape (plants in vases vs. lamps, closed lamp bodies vs. vases), and the vase class is small, so the decision boundary between vase and lamp is the weak point.
2. Segmentation Model 30 pts
Implement a PointNet segmentation model for chairs (6 part classes). Report the test accuracy, visualize at least 5 objects (including 2 bad predictions) next to their ground truth, report the accuracy of each, and interpret the results.
Model and training
The per-point encoder is split in two: a local encoder (3 → 64 → 64) and a global encoder (64 → 64 → 128 → 1024). The global features are max-pooled into one 1024-d vector per object, copied back to every point and concatenated with the point's 64-d local feature (1088-d per point). A per-point head (1088 → 512 → 256 → 128 → 6, BatchNorm + ReLU + dropout 0.3) predicts the part label. Same optimizer and batch size as Q1, trained for 100 epochs (~40 s/epoch). The best checkpoint is from epoch 79.
python train.py --task seg --num_epochs 100
python q2_seg_viz.py
Test accuracy: 89.16% of points
Per-object accuracy over the 617 test chairs: mean 89.2%, median 92.7%, min 47.5%, max 99.4%. Colors in the figures: class 2 (back), class 3 (arms), class 4 (legs / base), class 5 (seat), class 1 and class 0 (both rare; class 0 is white and nearly invisible on the white background). The part names are inferred from the visualizations.
| Part class | 0 | 1 | 2 (back) | 3 (arms) | 4 (legs) | 5 (seat) |
|---|---|---|---|---|---|---|
| Point accuracy | 89.7% | 29.3% | 91.6% | 75.8% | 88.0% | 91.1% |
Good predictions
The best object, the 25th-percentile object and the median object (left: ground truth, right: prediction).






Bad predictions
The two worst objects in the test set.




Interpretation
- Standard chairs are segmented almost perfectly. When back, seat, legs and arms are geometrically distinct (#471, #357, #205), the model gets 93–99%. The remaining errors are at part boundaries, for example where the arm meets the back or how thick the seat is in the office chair: the boundaries are slightly shifted, but every part is found.
- Box- and tub-shaped seats fail. Both bad cases are closed, upholstered shapes without legs. On the barrel chair (#351) the model labels the whole lower half of the cylinder as legs/base (blue), where the ground truth calls it back and arms; on the sofa (#235) it predicts a blue base and a red seat block, while the ground truth labels are scattered and mixed across the surface. Here the part boundaries are not geometric edges, so they are hard to infer, and the ground truth itself is ambiguous and noisy.
- Per-class accuracy follows how distinct a part is. Backs, seats and legs are above 88%; arms are lower (75.8%) because they touch both back and seat; class 1 is rare and poorly learned (29.3%). Vanilla PointNet has no local neighborhood features: each point only sees its own coordinates plus one global vector, so fine boundaries are its weak spot.
3. Robustness Analysis 20 pts
Conduct 2 experiments probing the robustness of the learned models, each on both the classification and segmentation tasks. Describe the procedure, report test accuracy and visualizations compared with Q1 & Q2, and interpret the results.
Both experiments evaluate the unchanged Q1 and Q2 checkpoints on the full test sets; nothing is retrained. Visualizations use the same objects throughout: chair #0, vase #617 and lamp #719 for classification, and chair #0 for segmentation.
python q3_robustness.py --axis z
Experiment 1: rotation
Procedure. Every test cloud is rotated about the z-axis by an angle
θ ∈ {0, 15, 30, 45, 60, 90, 135, 180}° (x ← x Rz(θ)T).
The y-axis is "up" in this dataset, so this tilts the object sideways in the viewing plane. The
training data is always upright and the models have no input transform or rotation augmentation.
| Rotation | 0° (Q1/Q2) | 15° | 30° | 45° | 60° | 90° | 135° | 180° |
|---|---|---|---|---|---|---|---|---|
| Classification | 97.69 | 93.60 | 73.66 | 39.03 | 30.54 | 24.76 | 23.92 | 67.37 |
| Segmentation | 89.16 | 83.64 | 71.57 | 61.30 | 52.41 | 39.34 | 30.40 | 40.87 |
Classification (predicted class under each figure)












Segmentation, chair #0 (top: GT, bottom: prediction)








Interpretation
- Small tilts are tolerated, large ones are not. At 15° both models lose only 4–6 points, but at 45° classification has fallen to 39% and segmentation to 61%. PointNet's shared MLP takes raw xyz coordinates, so its features are tied to the canonical upright pose seen during training. Without the T-Net or rotation augmentation nothing makes it rotation-invariant.
- Rotated objects collapse to "vase". At 60–135° classification accuracy is about 24–31%, below the 65% obtained by always guessing chair. In the sample objects, both the chair and the lamp are predicted as vase once tilted, while the vase stays correct. A tilted object no longer looks like an upright chair or lamp, and the model sends unfamiliar shapes to the vase class, which has the most varied shapes in training.
- Partial recovery at 180°. Classification recovers to 67%: an upside-down object has the same height and width as the upright one, and many objects are nearly left-right symmetric, so coarse features like "tall and thin" or "wide and short" still match. Segmentation does not recover as much (41%), because part labels depend on the vertical position (the seat is above the legs), which a 180° flip reverses.
- Segmentation degrades more gradually. At 30–90° it stays well above classification. Each point's label partly depends on local position relative to nearby parts, which survives a moderate tilt. In the 30° example the back and legs are still mostly correct and only the thin seat is partly labeled as back; at 90° (chair lying on its side) the labels follow height in the rotated frame instead of the part: the side that is now lowest becomes "legs" and the upper side "back".
Experiment 2: number of points per object
Procedure. Instead of all 10,000 points, each test object is evaluated on a random subset of N ∈ {5000, 2000, 1000, 500, 100, 50, 10} points (the same subset of indices for every object, fixed seed). Segmentation accuracy is measured on the kept points. The models were trained on 10,000 points.
| Points | 10,000 (Q1/Q2) | 5,000 | 2,000 | 1,000 | 500 | 100 | 50 | 10 |
|---|---|---|---|---|---|---|---|---|
| Classification | 97.69 | 97.69 | 96.96 | 96.64 | 96.43 | 90.77 | 70.51 | 26.02 |
| Segmentation | 89.16 | 89.27 | 89.18 | 88.75 | 87.72 | 82.11 | 79.03 | 71.10 |
Classification (predicted class under each figure)












Segmentation, chair #0 (top: GT, bottom: prediction)








Interpretation
- Very robust down to ~500 points. With 5% of the points, classification loses 1.3 points and segmentation 1.4. The global feature is a max over points, so it only depends on the few "critical" points that produce the maximum in each channel; a random subset of a dense cloud still contains points near those locations, and the overall shape is preserved.
- Classification breaks below ~100 points. At 50 points it drops to 71% and at 10 points to 26%, roughly chance level. With so few points, the max over each feature channel is taken over too few samples, so the global feature no longer resembles anything seen in training, and thin parts (lamp poles, chair legs) may not be sampled at all.
- Segmentation degrades more gently (79% at 50 points, 71% at 10). Each point's label mostly depends on its own position (e.g. height → legs vs. seat vs. back), which does not change when other points are removed; only the shared global feature gets worse. Part of the 10-point score also comes from the large classes: back and seat make up most of the points, so even coarse predictions score reasonably.
- In the visualizations, the chair keeps a recognizable structure down to 100 points and the predicted parts stay in the right places; at 50 points the cloud is hard to read even for a person, yet the model still labels back, seat and legs correctly for this chair.
4. Bonus: Locality with DGCNN 20 pts
Incorporate locality into the model (e.g. PointNet++, DGCNN, Point Transformer), train it on both tasks, and compare it with the vanilla PointNet quantitatively and qualitatively.
Model
I implemented DGCNN (Wang et al., 2019). Its building block is EdgeConv: for every point
xi it finds the k = 20 nearest neighbors, builds an edge feature
[xi, xj − xi] for each neighbor, passes it
through a shared MLP (Linear + BatchNorm + LeakyReLU 0.2), and takes the max over the neighbors.
xj − xi describes the local shape around the point, which
vanilla PointNet never sees. The graph is dynamic: the first layer uses xyz neighbors, the
later layers recompute kNN in feature space (pytorch3d.ops.knn_points /
knn_gather).
- Classification (1.80M parameters): EdgeConv 3→64→64→128→256; the four outputs are concatenated (512-d), lifted to 1024-d per point, then max- and mean-pooled (2048-d); head 2048→512 (dropout 0.5)→256→3.
- Segmentation (0.97M parameters): EdgeConv 3→64→64→64; the concatenated 192-d local feature is lifted to 1024-d and max-pooled into a global feature, which is copied back to every point and concatenated with the local feature (1216-d); head 1216→512 (dropout 0.5)→256→6.
The kNN search needs pairwise distances, which does not fit in memory at 10,000 points, so DGCNN trains on a random 1024-point subset of each object (a new subset every batch) and is tested on a fixed 1024-point subset (seed 0). Same optimizer as Q1/Q2 (Adam, lr 1e-3, batch 32). Classification: 75 epochs (~70 s/epoch), best checkpoint at epoch 59. Segmentation: 100 epochs (~24 s/epoch), best at epoch 82.
python train.py --task cls --model dgcnn --num_points 1024 --num_epochs 75
python train.py --task seg --model dgcnn --num_points 1024 --num_epochs 100
python q4_compare.py --num_points 1024
Quantitative comparison
For a fair comparison, PointNet is also evaluated on exactly the same 1024 points per object.
| Model | Points | Classification acc. | Segmentation acc. |
|---|---|---|---|
| PointNet (Q1/Q2) | 10,000 | 97.69% | 89.16% |
| PointNet | 1,024 | 97.38% | 88.67% |
| DGCNN | 1,024 | 98.22% | 91.36% |
Per class (PointNet at 1024 points → DGCNN at 1024 points):
| Classification | Chair | Vase | Lamp |
|---|---|---|---|
| Accuracy | 99.5% → 100.0% | 86.3% → 87.3% | 96.6% → 98.3% |
| Segmentation | 0 | 1 | 2 (back) | 3 (arms) | 4 (legs) | 5 (seat) |
|---|---|---|---|---|---|---|
| Point accuracy | 75.4% → 95.0% | 14.5% → 16.8% | 90.9% → 93.3% | 73.2% → 81.3% | 91.1% → 88.6% | 89.6% → 93.8% |
Confusion matrix of DGCNN (rows = ground truth chair / vase / lamp): [[617, 0, 0], [1, 89, 12], [0, 4, 230]]. It makes 17 errors where PointNet made 22.
Qualitative comparison: classification
The six PointNet failures from Q1, re-classified by DGCNN (colored by DGCNN's prediction). DGCNN fixes three of them: both chairs and the globe wall lamp.






Qualitative comparison: segmentation
The five Q2 objects: ground truth, PointNet (10,000 points), DGCNN (1024 points).















Robustness (Q3 experiments repeated on DGCNN)
| Rotation about z | 0° | 15° | 30° | 45° | 60° | 90° | 135° | 180° |
|---|---|---|---|---|---|---|---|---|
| Cls, PointNet | 97.7 | 93.6 | 73.7 | 39.0 | 30.5 | 24.8 | 23.9 | 67.4 |
| Cls, DGCNN | 98.2 | 96.8 | 85.9 | 56.7 | 31.6 | 26.1 | 24.1 | 36.0 |
| Seg, PointNet | 89.2 | 83.6 | 71.6 | 61.3 | 52.4 | 39.3 | 30.4 | 40.9 |
| Seg, DGCNN | 91.4 | 89.1 | 80.4 | 67.0 | 53.8 | 40.2 | 37.0 | 38.3 |
| Points per object | 1,000 / 1,024 | 500 | 100 | 50 | 10 |
|---|---|---|---|---|---|
| Cls, PointNet | 96.6 | 96.4 | 90.8 | 70.5 | 26.0 |
| Cls, DGCNN | 98.1 | 97.2 | 39.8 | 24.3 | 24.6 |
| Seg, PointNet | 88.8 | 87.7 | 82.1 | 79.0 | 71.1 |
| Seg, DGCNN | 91.4 | 90.3 | 71.1 | 55.4 | 43.5 |



Interpretation
- Locality helps, most of all on segmentation. On the same 1024 points, DGCNN improves classification by 0.8 points (97.4% → 98.2%) and segmentation by 2.7 points (88.7% → 91.4%), and with ~10× fewer points it beats PointNet run on all 10,000. Part labels are decided at boundaries (arm vs. back, seat vs. back), and the edge features xj − xi directly encode the local surface shape, so boundaries become sharper: arms go from 73% to 81% and class 0 from 75% to 95%. Class 1 stays poor (17%); it is too rare for either model to learn.
- Classification: local detail fixes the "thin structure" failures. The two tall chairs that PointNet called lamps are recognized from local cues such as seat-and-leg junctions, and the globe wall lamp is recognized as a lamp. The remaining failures (plants in pots called lamps, a closed lantern called a vase) are ambiguous even for a person, so locality cannot fix them.
- Hard segmentation cases stay hard. The sofa (#235) and barrel chair (#351) stay around 50% with both models: their part boundaries are not geometric edges and the ground truth is noisy, so better local geometry gives no extra information.
- Rotation: slightly more robust at small angles, not invariant. DGCNN degrades more slowly up to 45° (e.g. cls 85.9% vs 73.7% at 30°) because relative offsets between neighbors change less than absolute coordinates, but every layer still includes xi, so large rotations break it just as badly.
- Point count: DGCNN is much less robust to very sparse clouds. Below 500 points it collapses (39.8% cls at 100 points, vs 90.8% for PointNet). The neighborhood size is fixed at k = 20 points, so on a sparse cloud each "local" neighborhood spans a large part of the object (at 10 points it is larger than the object). The edge features then have a very different scale than in training. PointNet processes each point alone and does not have this problem. Training with a varying number of points, or using radius-based neighborhoods as in PointNet++, would address this.