Point Cloud Processing¶
16-825 Learning for 3D: Assignment 5
Author: Benny Zhu (haozhenz)
Date: November 2026
Failure Cases¶
| Chair | Vase | Lamp | |
|---|---|---|---|
| Point Cloud | ![]() |
![]() |
![]() |
| Predicted | lamp | lamp | vase |
| Ground Truth | chair | vase | lamp |
All three failures happen when an object's global silhouette is closer to another class than to its own. Chair is almost always right (99.8%) because chairs have a very distinctive structure. Most of the errors are lamp and vase being mistaken for each other, since both are often a round, vertically elongated object on a narrow base.
Bad Predictions¶
| Object | Ground Truth | Prediction | Accuracy |
|---|---|---|---|
| #26 | ![]() |
![]() |
42.0% |
| #163 | ![]() |
![]() |
32.6% |
The model reliably segments the large, geometrically distinct parts: the legs or base (blue), the seat (red), and the backrest (cyan) are correct on all three random chairs, including unusual designs such as the egg chair's pedestal base. Most of the errors come from the armrests (yellow). Armrests sit at the same height as the seat or back and touch both, so the model often assigns them to a neighbor.
These failure modes match how PointNet works. Each point is classified from its own coordinates plus a single global feature, so the predictions mainly follow height and overall position. The model has no way to use local neighborhood geometry to find where one part ends and the next begins. It works well when parts are clearly separated in space, and breaks down for touching parts (armrests), rare classes, and unusual shapes. The two worst objects (#26, #163) are boxy sofa-like chairs where all three problems occur together, and the model essentially splits them horizontally by height.
3. Robustness Analysis (20 points)¶
3.1 Experiment 1: Rotation (10 points)¶
python eval_cls.py --rot_deg <angle>
python eval_seg.py --rot_deg <angle>
Procedure: I rotated every test point cloud around the z-axis by 15°, 30°, 45°, 60°, and 90° and evaluated the trained Q1 and Q2 models on the rotated inputs without retraining. The same 10,000 sampled points were used in every run, so rotation is the only change compared with the Q1/Q2 baselines.
| Rotation Angle | Classification Accuracy | Segmentation Accuracy |
|---|---|---|
| 0° (Q1 / Q2 baseline) | 97.27% | 88.35% |
| 15° | 93.91% | 83.16% |
| 30° | 66.53% | 70.57% |
| 45° | 28.65% | 58.90% |
| 60° | 23.92% | 51.40% |
| 90° | 24.24% | 40.60% |
Classification visualizations:
| 0° | 30° | 60° | 90° |
|---|---|---|---|
![]() |
![]() |
![]() |
![]() |
| Pred: chair | Pred: chair | Pred: vase | Pred: vase |
Segmentation visualizations:
| 0° | 30° | 60° | 90° | |
|---|---|---|---|---|
| Ground Truth | ![]() |
![]() |
![]() |
![]() |
| Prediction | ![]() |
![]() |
![]() |
![]() |
| Accuracy | 83.7% | 57.5% | 38.6% | 3.5% |
Both models are very sensitive to rotation. Accuracy falls steadily as the angle grows. All training objects are upright in the same canonical pose, and we use no rotation augmentation and no T-Net. As a result, the features PointNet learns are tied to absolute coordinates rather than to the object's shape: the tilted chair in the example is still a chair at 30°, but it is predicted as a vase at 60° and 90°. Segmentation fails in the same way. The per-point predictions still follow the original vertical axis: whatever ends up near the top is labeled back, the middle seat, and the bottom legs, regardless of which part a point actually belongs to. So accuracy on the example chair falls from 83.7% to 3.5% at 90°.
3.2 Experiment 2: Number of Points (10 points)¶
python eval_cls.py --num_points <N>
python eval_seg.py --num_points <N>
Procedure: I evaluated the trained Q1 and Q2 models on test point clouds randomly subsampled to 1,000, 100, and 50 points per object, compared with 10,000 points in the original setting. No retraining was done, and the same subset of point indices was used for every object (and as the labels in segmentation).
| Number of Points | Classification Accuracy | Segmentation Accuracy |
|---|---|---|
| 10000 (Q1 / Q2 baseline) | 97.27% | 88.35% |
| 5000 | 97.27% | 88.39% |
| 1000 | 97.38% | 88.23% |
| 500 | 96.96% | 87.74% |
| 100 | 91.40% | 81.71% |
| 50 | 76.29% | 75.02% |
Classification visualizations:
| 10000 | 1000 | 100 | 50 |
|---|---|---|---|
![]() |
![]() |
![]() |
![]() |
| Pred: chair | Pred: chair | Pred: lamp | Pred: lamp |
Segmentation visualizations:
| 10000 | 1000 | 100 | 50 | |
|---|---|---|---|---|
| Ground Truth | ![]() |
![]() |
![]() |
![]() |
| Prediction | ![]() |
![]() |
![]() |
![]() |
| Accuracy | 95.44% | 98.90% | 30.00% | 20.00% |
PointNet is much more robust to fewer points than to rotation. With only 10% of the points (1,000) accuracy is essentially unchanged, and even with 100 points it stays above 80% for both tasks. This follows from the max-pool design. The global feature is determined by a small set of "critical points" that give the maximum response in each channel, so as long as the subsampled cloud still outlines the overall shape, the feature barely changes. Performance only drops sharply at 50 points, when thin structures such as legs and the backrest are covered by just a few points or disappear. In the classification example, the remaining points of the office chair cluster around the seat and the central pole, and that sparse silhouette looks more like a lamp. In the segmentation example, the model mostly loses the rare, thin parts, and with so few points each mistake costs more accuracy.


































