Assignment 5 – Point Cloud Processing
📘 Quick Navigation
Q1. Classification Model (40 pts)
Test Accuracy: 97.06%
Correct Predictions
Chair
Vase
Lamp
Wrong Predictions
Pred: Object 543
Pred: Object 671
Pred: Object 758
Interpretation: The errors indicate weaknesses in handling atypical or ambiguous shapes, especially between Lamps and Vases due to similar vertical profiles.
Q2. Segmentation Model (40 pts)
Test Accuracy: 89.82%
Good Predictions
Sample 15
Sample 65
Sample 437
Relatively Bad Predictions
Sample 406
Sample 495
Sample 452
Interpretation: Errors arise primarily on chairs with unusual geometry → the model struggles with out-of-distribution samples.
Q3. Robustness Analysis (20 pts)
Classification – Rotation Sensitivity
| Rotation | Accuracy |
|---|---|
| 0° | 97.06% |
| 15° | 94.54% |
| 30° | 78.07% |
| 45° | 63.06% |
| 90° | 70.62% |
Chair – Rotation Examples
0°
15°
30°
45°
90°
Lamp – Rotation Examples
0°
15°
30°
45°
90°
Vase – Rotation Examples
0°
15°
30°
45°
90°
Interpretation: The model is highly sensitive to rotations because PointNet-like architectures are not rotation equivariant. Large drops occur at 30–90°, especially for elongated or asymmetric objects.
Classification – Number of Points
| Points | Accuracy |
|---|---|
| 512 | 95.80% |
| 1024 | 96.64% |
| 2048 | 96.64% |
| 4096 | 97.17% |
| 10000 | 97.06% |
Chair – Points Comparison
512
1024
2048
4096
10k
Lamp – Points Comparison
512
1024
2048
4096
10k
Vase – Points Comparison
512
1024
2048
4096
10k
Interpretation: Removing points slightly reduces structural detail but the classifier remains fairly robust due to global feature aggregation.
Segmentation – Rotation Sensitivity
| Rotation | Accuracy |
|---|---|
| 0° | 89.82% |
| 15° | 86.72% |
| 30° | 79.34% |
| 45° | 71.45% |
| 90° | 54.76% |
Segmentation – Chair 1
GT
0°
15°
30°
45°
90°
Segmentation – Chair 2
GT
0°
15°
30°
45°
90°
Interpretation: Segmentation collapses heavily at 90° because the model learned orientation-dependent part layout rather than true geometric part structure.
Segmentation – Number of Points
| Points | Accuracy |
|---|---|
| 512 | 88.60% |
| 1024 | 89.39% |
| 2048 | 89.61% |
| 4096 | 89.79% |
| 10000 | 89.82% |
Segmentation – Points Sparsity 1
GT
512
1024
2048
4096
10k
Segmentation – Points Sparsity 2
GT
512
1024
2048
4096
10k
Interpretation: Segmentation is robust to sparsity because local neighborhood patterns remain mostly intact even with fewer points.
Q4. Bonus – Locality (20 pts)
DGCNN vs Q-models (Rotation)
| Experiment | 0° | 15° | 30° | 45° | 90° |
|---|---|---|---|---|---|
| DGCNN Classification | 98.32% | 97.90% | 91.82% | 75.76% | 65.58% |
| DGCNN Segmentation | 91.75% | 90.17% | 86.50% | 82.35% | 63.53% |
| Q2 Classification | 97.48% | 95.28% | 86.57% | 71.67% | 75.66% |
| Q3 Segmentation | 89.70% | 85.43% | 78.29% | 70.17% | 54.95% |
Points Comparison
| Experiment | 512 pts | 1024 pts |
|---|---|---|
| DGCNN Classification | 97.38% | 98.43% |
| DGCNN Segmentation | 90.33% | 91.54% |
| Q2 Classification | 96.54% | 97.38% |
| Q3 Segmentation | 89.42% | 89.71% |
Interpretation: DGCNN outperforms the Q-models in rotation robustness and point sparsity, thanks to local neighborhood feature aggregation (k-NN graph construction).
DGCNN Visualizations
Chair
Vase
Lamp
