16-825: Learning for 3D Vision

Assignment 5

Fall 2026

Point Cloud Classification and Segmentation

Contents

1. Classification Model

1. Classification

30 points

Test accuracy: 98.01%

Random test samples

Test 233
#233 · predicted chair
Test 636
#636 · predicted vase
Test 213
#213 · predicted chair
Test 222
#222 · predicted chair
Test 406
#406 · predicted lamp (GT chair)
Test 496
#496 · predicted chair

Failure cases

Failure chair
GT chair · predicted lamp
Failure vase
GT vase · predicted lamp
Failure lamp
GT lamp · predicted vase

All three just look like the other class. The chair is basically a flat panel, the vase sits on a tall pedestal like a floor lamp, and the lamp is a plain cylinder with no shade. Vases fail the most since there are few of them in training and they share the tall, round shape with lamps.

2. Segmentation Model

2. Segmentation

30 points

Test accuracy: 90.29%

Ground truth (left) vs. prediction (right)

586 GT
#586 · GT
586 prediction
#586 · 95.04%
418 GT
#418 · GT
418 prediction
#418 · 95.37%
240 GT
#240 · GT
240 prediction
#240 · 92.76%
235 GT
#235 · GT
235 prediction
#235 · 47.13% (bad)
605 GT
#605 · GT
605 prediction
#605 · 47.14% (bad)

Normal chairs are mostly right, with errors only along part boundaries (e.g. the seat edge). The two bad ones are odd chairs: #235 is a boxy sofa where seat, back and arms blend together, and #605 has a sled-like base that gets labeled as seat/arms instead of legs. The model seems to lean on height to pick the part.

3. Robustness Analysis

3.1 Rotation

10 points

Procedure. Rotate the test point clouds around the vertical axis by different angles and evaluate the Q1/Q2 models.

AngleClassificationSegmentation
0° (Q1/Q2)98.01%90.29%
15°96.96%86.27%
30°80.59%79.11%
45°64.74%72.97%
60°58.45%66.67%
90°57.19%58.37%

Classification

rot 0
0° · chair
rot 30
30° · chair
rot 60
60° · vase
rot 90
90° · vase
rot 0
0° · vase
rot 30
30° · vase
rot 60
60° · vase
rot 90
90° · vase

Segmentation

seg rot 0
0° · 95.04%
seg rot 30
30° · 69.50%
seg rot 60
60° · 77.99%
seg rot 90
90° · 63.19%

Not robust to rotation. All training shapes face the same way and PointNet uses raw xyz with no T-Net or augmentation, so a rotated chair looks like a new shape. Segmentation drops a bit slower since rotating around the vertical axis keeps each point's height.

3.2 Number of Points

10 points

Procedure. Randomly sample fewer points per object at test time (--num_points) and evaluate the Q1/Q2 models.

PointsClassificationSegmentation
10000 (Q1/Q2)98.01%90.29%
500097.90%90.32%
100097.06%89.42%
50096.85%87.85%
10092.86%79.65%

Classification

np 10000
10000 · chair
np 1000
1000 · chair
np 100
100 · chair
np 10000
10000 · vase
np 1000
1000 · vase
np 100
100 · vase

Segmentation

seg np 10000
10000 · 95.04%
seg np 1000
1000 · 96.10%
seg np 100
100 · 74.00%

Pretty robust. Max pooling only depends on a few key points, and a few hundred random points still cover the shape. At 100 points segmentation suffers more because thin parts like legs end up with just a few points.

4. Bonus: Locality

4. DGCNN

Bonus · 20 points

Model. DGCNN with EdgeConv layers (k = 20). Each layer finds the k nearest neighbors in feature space, uses [xi, xj − xi] as edge features, applies an MLP and max-pools over neighbors. Trained and tested with 1024 points (k-NN memory is O(N²)), so PointNet is also tested at 1024 points below.

ModelClassificationSegmentation
PointNet (Q1/Q2, 10000 pts)98.01%90.29%
PointNet (Q1/Q2, 1024 pts)97.27%89.51%
DGCNN (1024 pts)98.01%91.21%

GT / PointNet / DGCNN

235 GT
#235 · GT
235 PointNet
PointNet · 46.68%
235 DGCNN
DGCNN · 50.10%
605 GT
#605 · GT
605 PointNet
PointNet · 63.67%
605 DGCNN
DGCNN · 77.25%
586 GT
#586 · GT
586 PointNet
PointNet · 97.46%
586 DGCNN
DGCNN · 99.02%

DGCNN fixes two of the three Q1 failures (#406, #650); the cylinder lamp #803 is still called a vase.