Assignment 5: Point Cloud Classification and Segmentation

1. Classification Model

Test accuracy: 97.90% (933 / 953). Per class: chair 100.0%, vase 88.2%, lamp 96.6%.

Random test objects

Failure cases

No chair was misclassified, so the chair shown is the one with the lowest predicted chair probability.

2. Segmentation Model

Test accuracy: 89.85% of all test points. Left: ground truth, right: prediction, with per-object accuracy.

3. Robustness Analysis

Experiment 1: rotation

Procedure. Rotate every test cloud about the x axis (tipping it backwards) by a fixed angle, with no retraining (--rotate).

rotation0° (Q1/Q2)15°30°45°60°90°180°
classification97.9%94.8%82.4%48.8%33.1%28.9%66.4%
segmentation89.9%82.6%71.3%62.5%43.5%26.9%32.2%

PointNet uses raw coordinates and has no T-Net or rotation augmentation, so accuracy holds up to about 15° and then drops fast. Tipped chairs are mostly called lamps, and segmentation labels points by height (bottom = legs, middle = seat, top = back) whatever the actual part.

Experiment 2: number of points

Procedure. Keep the same random n of the 10,000 points for every test object (--num_points).

points10,000 (Q1/Q2)5,0001,00050010050
classification97.9%97.8%96.8%96.5%94.3%85.5%
segmentation89.9%89.9%89.7%88.8%82.3%78.1%

Accuracy barely changes down to 1,000 points: the global feature is a max over points, and a few points per region already reach the same max. Below a few hundred points thin parts lose most of their points, so segmentation degrades first.

4. Bonus: Locality with PointNet++

Model. PointNet++ with single-scale grouping, in plain PyTorch. Three set-abstraction levels (farthest point sampling of 512 and 128 centers, ball query with radius 0.2 / 0.4, and a small PointNet on each neighborhood in relative coordinates, then one global level) build features from local neighborhoods. Segmentation adds feature propagation (3-nearest-neighbor interpolation with skip connections) back to every point. Trained and tested on 2,048 points per object; PointNet is also evaluated on the same 2,048 points for comparison.

modelclassificationsegmentation
PointNet (Q1/Q2), 10,000 points97.90%89.85%
PointNet (Q1/Q2), 2,048 points97.59%90.00%
PointNet++, 2,048 points97.80%91.25%

Segmentation improves by 1.25 points (better on 426 of 617 chairs), most on hard shapes such as sofa #235, because each part label now comes from the local neighborhood instead of mainly from height. Classification is essentially tied: PointNet++ fixes some vase→lamp errors (#629, #651) but adds others (#290).