16-825 Assignment 5: Point Cloud Classification and Segmentation

Jerry Hou (jerryhou)

All test accuracies use the full 10,000 points per object unless stated otherwise. Class labels: 0 = chair, 1 = vase, 2 = lamp. Segmentation colours follow viz_seg: white, magenta, cyan, yellow, blue and red for parts 0–5.

Q1. Classification Model

PointNet without the transformation blocks: a shared per-point MLP (3 → 64 → 128 → 1024), then max-pooling over points, then an FC head (1024 → 512 → 3).

Test accuracy: 97.27%

Random test samples

Object 39 Object 256 Object 293
GT chair, pred chair GT chair, pred chair GT chair, pred chair
Object 485 Object 604 Object 806
GT chair, pred chair GT chair, pred chair GT lamp, pred lamp

Failure cases (one per class)

Chair → predicted lamp (406) Vase → predicted lamp (618) Lamp → predicted vase (726)

Errors by true class: 1 chair, 16 vases and 9 lamps are misclassified.

Vases and lamps cause almost all the errors. Both are tall shapes with rotational symmetry, so a single global max-pooled feature cannot easily tell them apart. Telling them apart needs local cues, such as a thin stem or a hollow opening, that the max-pool can wash out.

Q2. Segmentation Model

This uses the same per-point encoder. The 64-d local feature of each point is concatenated with the 1024-d global feature, then a per-point MLP (1088 → 512 → 256 → 128 → 6) predicts each point’s part.

Test accuracy: 89.95% (fraction of points labelled correctly)

Object Ground truth Prediction Accuracy
138 97.1%
428 96.5%
552 98.7%
255 (bad) 43.8%
351 (bad) 47.2%

Objects 138, 428 and 552 are random picks from the better half of the test set. Objects 255 and 351 are the two worst objects in the test set.

Interpretation. On typical chairs with a clear back, seat and legs, the model is above 96%. Its errors sit mostly at the boundaries between parts, for example where the seat meets the legs. The two failures are both unusual, boxy shapes:

In both cases the model uses where a point sits on the chair as a stand-in for part identity. That works for typical chairs and fails when the layout is unusual.

Q3. Robustness Analysis

Both experiments use the Q1/Q2 checkpoints without retraining. The 0° and 10,000-point rows are the Q1/Q2 baselines. The visualisations use test object 0, a chair, for both tasks. Classification predictions are given as labels (0 = chair, 2 = lamp).

Experiment 1: rotation

Procedure. I rotate every test point cloud about the x-axis. The y-axis is “up”, so this tips objects forward. Angles: 0°, 15°, 30°, 45°, 60°, 90° and 180°. Then I re-evaluate with python eval_cls.py --rot_deg D and python eval_seg.py --rot_deg D.

Rotation Cls accuracy Seg accuracy
0° (Q1/Q2) 97.3% 90.0%
15° 91.1% 83.6%
30° 68.8% 70.6%
45° 36.8% 59.9%
60° 27.1% 44.9%
90° 25.4% 29.0%
180° 65.1% 30.6%
Rotation Classification (object 0) Seg ground truth Seg prediction
0°
pred chair ✓
30°
pred lamp ✗
90°
pred lamp ✗
180°
pred chair ✓

Interpretation.

Experiment 2: number of points per object

Procedure. I evaluate with --num_points set to 10000, 5000, 1000, 500, 100, 50 and 10. Each run draws a fixed random subset of points for all objects (seed 0).

Points Cls accuracy Seg accuracy
10000 (Q1/Q2) 97.3% 90.0%
5000 97.2% 89.9%
1000 97.0% 88.9%
500 96.5% 87.0%
100 94.6% 78.1%
50 91.1% 73.6%
10 72.9% 64.2%
Points Classification (object 0) Seg ground truth Seg prediction
10000
pred chair ✓
500
pred chair ✓
50
pred chair ✓
10
pred chair ✓

Interpretation.

Q4. Bonus: Locality with PointNet++

Model. I implemented PointNet++ with single-scale grouping in plain PyTorch (models.py), for both tasks.

Task PointNet (Q1/Q2) PointNet++
Classification 97.27% 98.64%
Segmentation 89.95% 91.54%

In classification, PointNet++ fixes 18 of PointNet’s 26 test errors and introduces 5 new ones. All three Q1 failure cases from above are now correct:

Object PointNet PointNet++

406 (chair)
lamp ✗ chair ✓

618 (vase)
lamp ✗ vase ✓

726 (lamp)
vase ✗ lamp ✓

Segmentation, on the same objects as Q2:

Object Ground truth PointNet PointNet++
138
97.1%

96.9%
552
98.7%

98.1%
255
43.8%

67.1%
351
47.2%

55.3%

Interpretation.