Assignment 5: PointNet Classification and Segmentation

Name: Tashvik Dhamija (tdhamija)

1. Classification Model

python train.py --task cls

python eval_cls.py

Test accuracy: 97.38%

Random Test Predictions

GT: chair, Pred: chair
GT: chair, Pred: chair
GT: chair, Pred: chair

Failure Cases (1 per class)

GT: chair, Pred: lamp
GT: vase, Pred: lamp
GT: lamp, Pred: vase

Lamps and vases often have similar shapes. Since the model only learns from point positions, with no other information about the object, it can get confused between them.

The chair it fails on is folded, so its point cloud is concentrated like a lamp or vase, instead of spread out like a normal chair point cloud. That's why it gets it wrong.

2. Segmentation Model

python train.py --task seg

python eval_seg.py

Test accuracy: 89.90%

Segmentation color legend (applies to every segmentation visualization on this page):

Class 0 Class 1 Class 2 Class 3 Class 4 Class 5

Segmentation Visualizations (5 objects, 2 bad predictions)

GT, accuracy 99.63%
Pred, accuracy 99.63%
GT, accuracy 99.60%
Pred, accuracy 99.60%
GT, accuracy 99.70%
Pred, accuracy 99.70%
GT, accuracy 46.50% (bad)
Pred, accuracy 46.50% (bad)
GT, accuracy 46.06% (bad)
Pred, accuracy 46.06% (bad)

The model does well on samples that look like a common chair. But for chair types with a different shape, or an extra element like a futon-style chair, it struggles, probably because it doesn't see enough samples of those shapes during training.

3. Robustness Analysis

3.1 Rotation

RotationClassification accuracySegmentation accuracy
0°97.38%89.90%
15°94.12%86.74%
30°84.89%80.55%
45°70.83%73.15%
90°63.80%58.28%
180°31.58%48.27%

Procedure: rotated the input point clouds around the up (Y) axis by each angle at evaluation time, using the Q1/Q2 checkpoints as is, and measured how accuracy changes.

Classification

0°, Pred: chair (correct)
15°, Pred: chair (correct)
30°, Pred: chair (correct)
45°, Pred: vase (wrong)
90°, Pred: lamp (wrong)
180°, Pred: lamp (wrong)

Segmentation

Class 0 Class 1 Class 2 Class 3 Class 4 Class 5
0° GT
0° Pred (94.01%)
90° GT
90° Pred (79.19%)
180° GT
180° Pred (47.88%)

Since the model only learns to use spatial info, rotation puts parts of the object in areas the model doesn't expect, which confuses it. The model never learns to deal with rotation at all.

Classification suffers more than segmentation, since it relies on one globally pooled feature for the whole object. Once that pooled feature is thrown off by rotation, there's nothing else to fall back on. Segmentation still has local per-point features to work with, so it holds up a bit better.

3.2 Reduced Number of Points

PointsClassification accuracySegmentation accuracy
500096.96%89.85%
200096.43%89.32%
100096.43%88.40%
50096.43%86.55%
10093.18%79.72%
5089.82%76.25%

Procedure: randomly subsampled the input point clouds down to each point count at evaluation time, using the Q1/Q2 checkpoints as is, and measured how accuracy changes.

Classification

5000 pts, Pred: chair (correct)
2000 pts, Pred: chair (correct)
1000 pts, Pred: chair (correct)
500 pts, Pred: chair (correct)
100 pts, Pred: chair (correct)
50 pts, Pred: chair (correct)

Segmentation

Class 0 Class 1 Class 2 Class 3 Class 4 Class 5
5000 pts GT
5000 pts Pred (94.22%)
500 pts GT
500 pts Pred (92.80%)
50 pts GT
50 pts Pred (86.00%)

Classification doesn't suffer much from reducing points. Even with as few as 50 points, the model can still identify the global structure of the object from the pooled features.

Segmentation needs enough point density to tell apart where one part ends and another begins. With very few points, there isn't enough local detail left near those part boundaries, so segmentation accuracy drops more than classification. Classification only needs the rough global shape, which survives sparsity fine, segmentation needs fine per-point detail, which doesn't.

4. Bonus: Locality (DGCNN)

Implemented DGCNN (Dynamic Graph CNN). For each point, we find its k=20 nearest neighbors and build a local graph. Three EdgeConv layers (3 to 64 to 128 to 256 channels) learn from each point and its neighbor offsets, then max-pool over the neighbors. The classification variant adds a global max-pool and FC head, same as cls_model. The segmentation variant applies an FC head directly on the per-point features, same as seg_model.

Finding neighbors this way gets expensive fast as point count grows, so we first subsample the input down to 1024 points for classification and 2048 points for segmentation using farthest point sampling (pytorch3d's sample_farthest_points). This matches the point count used in the original DGCNN paper.

python train.py --task cls --model_type dgcnn --num_points 1024 --batch_size 8

python train.py --task seg --model_type dgcnn --num_points 2048 --batch_size 8

python eval_cls.py --model_type dgcnn --num_points 1024

python eval_seg.py --model_type dgcnn --num_points 2048

TaskPointNet (Q1/Q2)DGCNN (Q4)
Classification97.38%97.17%
Segmentation89.90%91.60%

Classification: DGCNN Random Predictions

GT: lamp, Pred: lamp
GT: chair, Pred: chair
GT: chair, Pred: chair

Classification: DGCNN Failure Cases

GT: vase, Pred: lamp
GT: lamp, Pred: vase

No chair failure case was found in this test run (DGCNN classified every chair in the batch correctly).

Segmentation: DGCNN Visualizations

Class 0 Class 1 Class 2 Class 3 Class 4 Class 5
GT, accuracy 99.71%
Pred, accuracy 99.71%
GT, accuracy 99.71%
Pred, accuracy 99.71%
GT, accuracy 50.44% (bad)
Pred, accuracy 50.44% (bad)

Classification: DGCNN gets almost the same accuracy (97.17% vs 97.38%) using far fewer points, 1024 instead of 10000. This shows it needs much less data to reach the same level of accuracy.

Segmentation: DGCNN does better (91.60% vs 89.90%) because it builds better local context through EdgeConv, which helps distinguish nearby points near part boundaries.

Limitation: DGCNN is more expensive. Building the neighbor graph and running EdgeConv adds extra computation, so it has higher complexity than plain PointNet.