1. Classification Model 30 pts

Implement a PointNet classifier for chairs, vases and lamps. Report the test accuracy, visualize a few random test point clouds with their predicted classes, and visualize at least one failure for each class with an interpretation.

Model and training

Vanilla PointNet without the transformation blocks. A shared per-point MLP (3 → 64 → 64 → 64 → 128 → 1024, each layer Linear + BatchNorm + ReLU) encodes every point independently; a max-pool over the points gives a 1024-d global feature that is invariant to point order; a head (1024 → 512 → 256 → 3, BatchNorm + ReLU + dropout 0.3 on the hidden layers) outputs class logits. Trained with cross-entropy, Adam (lr 1e-3), batch size 32, all 10,000 points per object, for 75 epochs (~20 s/epoch on a 16 GB laptop GPU). The best checkpoint is from the last epoch, 74.

python train.py --task cls --num_epochs 75
python q1_cls_viz.py

Test accuracy: 97.69% (931 / 953)

Almost all errors are vase ↔ lamp confusions. Vases are the smallest class (11% of the test set, and similarly rare in training), so they have the lowest accuracy.

Random test samples

Two random objects per class (sampling uniformly would give mostly chairs, which are 65% of the test set). Each cloud is colored by its predicted class: chair, vase, lamp. All six are classified correctly.

Numbers in parentheses are softmax confidences. Chairs and lamps are classified with near certainty; vases, even when correct, get noticeably lower confidence.

Failure cases

The two most confident mistakes for each ground-truth class (chairs only have two errors in total). Colors again show the predicted class.

Interpretation

2. Segmentation Model 30 pts

Implement a PointNet segmentation model for chairs (6 part classes). Report the test accuracy, visualize at least 5 objects (including 2 bad predictions) next to their ground truth, report the accuracy of each, and interpret the results.

Model and training

The per-point encoder is split in two: a local encoder (3 → 64 → 64) and a global encoder (64 → 64 → 128 → 1024). The global features are max-pooled into one 1024-d vector per object, copied back to every point and concatenated with the point's 64-d local feature (1088-d per point). A per-point head (1088 → 512 → 256 → 128 → 6, BatchNorm + ReLU + dropout 0.3) predicts the part label. Same optimizer and batch size as Q1, trained for 100 epochs (~40 s/epoch). The best checkpoint is from epoch 79.

python train.py --task seg --num_epochs 100
python q2_seg_viz.py

Test accuracy: 89.16% of points

Per-object accuracy over the 617 test chairs: mean 89.2%, median 92.7%, min 47.5%, max 99.4%. Colors in the figures: class 2 (back), class 3 (arms), class 4 (legs / base), class 5 (seat), class 1 and class 0 (both rare; class 0 is white and nearly invisible on the white background). The part names are inferred from the visualizations.

Part class012 (back)3 (arms)4 (legs)5 (seat)
Point accuracy89.7%29.3%91.6%75.8%88.0%91.1%

Good predictions

The best object, the 25th-percentile object and the median object (left: ground truth, right: prediction).

Bad predictions

The two worst objects in the test set.

Interpretation

3. Robustness Analysis 20 pts

Conduct 2 experiments probing the robustness of the learned models, each on both the classification and segmentation tasks. Describe the procedure, report test accuracy and visualizations compared with Q1 & Q2, and interpret the results.

Both experiments evaluate the unchanged Q1 and Q2 checkpoints on the full test sets; nothing is retrained. Visualizations use the same objects throughout: chair #0, vase #617 and lamp #719 for classification, and chair #0 for segmentation.

python q3_robustness.py --axis z

Experiment 1: rotation

Procedure. Every test cloud is rotated about the z-axis by an angle θ ∈ {0, 15, 30, 45, 60, 90, 135, 180}° (x ← x Rz(θ)T). The y-axis is "up" in this dataset, so this tilts the object sideways in the viewing plane. The training data is always upright and the models have no input transform or rotation augmentation.

Rotation0° (Q1/Q2)15°30°45°60°90°135°180°
Classification97.6993.6073.6639.0330.5424.7623.9267.37
Segmentation89.1683.6471.5761.3052.4139.3430.4040.87

Classification (predicted class under each figure)

Segmentation, chair #0 (top: GT, bottom: prediction)

Interpretation

Experiment 2: number of points per object

Procedure. Instead of all 10,000 points, each test object is evaluated on a random subset of N ∈ {5000, 2000, 1000, 500, 100, 50, 10} points (the same subset of indices for every object, fixed seed). Segmentation accuracy is measured on the kept points. The models were trained on 10,000 points.

Points10,000 (Q1/Q2)5,0002,0001,0005001005010
Classification97.6997.6996.9696.6496.4390.7770.5126.02
Segmentation89.1689.2789.1888.7587.7282.1179.0371.10

Classification (predicted class under each figure)

Segmentation, chair #0 (top: GT, bottom: prediction)

Interpretation

4. Bonus: Locality with DGCNN 20 pts

Incorporate locality into the model (e.g. PointNet++, DGCNN, Point Transformer), train it on both tasks, and compare it with the vanilla PointNet quantitatively and qualitatively.

Model

I implemented DGCNN (Wang et al., 2019). Its building block is EdgeConv: for every point xi it finds the k = 20 nearest neighbors, builds an edge feature [xi, xj − xi] for each neighbor, passes it through a shared MLP (Linear + BatchNorm + LeakyReLU 0.2), and takes the max over the neighbors. xj − xi describes the local shape around the point, which vanilla PointNet never sees. The graph is dynamic: the first layer uses xyz neighbors, the later layers recompute kNN in feature space (pytorch3d.ops.knn_points / knn_gather).

The kNN search needs pairwise distances, which does not fit in memory at 10,000 points, so DGCNN trains on a random 1024-point subset of each object (a new subset every batch) and is tested on a fixed 1024-point subset (seed 0). Same optimizer as Q1/Q2 (Adam, lr 1e-3, batch 32). Classification: 75 epochs (~70 s/epoch), best checkpoint at epoch 59. Segmentation: 100 epochs (~24 s/epoch), best at epoch 82.

python train.py --task cls --model dgcnn --num_points 1024 --num_epochs 75
python train.py --task seg --model dgcnn --num_points 1024 --num_epochs 100
python q4_compare.py --num_points 1024

Quantitative comparison

For a fair comparison, PointNet is also evaluated on exactly the same 1024 points per object.

ModelPointsClassification acc.Segmentation acc.
PointNet (Q1/Q2)10,00097.69%89.16%
PointNet1,02497.38%88.67%
DGCNN1,02498.22%91.36%

Per class (PointNet at 1024 points → DGCNN at 1024 points):

ClassificationChairVaseLamp
Accuracy99.5% → 100.0%86.3% → 87.3%96.6% → 98.3%
Segmentation012 (back)3 (arms)4 (legs)5 (seat)
Point accuracy75.4% → 95.0%14.5% → 16.8%90.9% → 93.3%73.2% → 81.3%91.1% → 88.6%89.6% → 93.8%

Confusion matrix of DGCNN (rows = ground truth chair / vase / lamp): [[617, 0, 0], [1, 89, 12], [0, 4, 230]]. It makes 17 errors where PointNet made 22.

Qualitative comparison: classification

The six PointNet failures from Q1, re-classified by DGCNN (colored by DGCNN's prediction). DGCNN fixes three of them: both chairs and the globe wall lamp.

Qualitative comparison: segmentation

The five Q2 objects: ground truth, PointNet (10,000 points), DGCNN (1024 points).

Robustness (Q3 experiments repeated on DGCNN)

Rotation about z0°15°30°45°60°90°135°180°
Cls, PointNet97.793.673.739.030.524.823.967.4
Cls, DGCNN98.296.885.956.731.626.124.136.0
Seg, PointNet89.283.671.661.352.439.330.440.9
Seg, DGCNN91.489.180.467.053.840.237.038.3
Points per object1,000 / 1,0245001005010
Cls, PointNet96.696.490.870.526.0
Cls, DGCNN98.197.239.824.324.6
Seg, PointNet88.887.782.179.071.1
Seg, DGCNN91.490.371.155.443.5

Interpretation