Point Cloud Processing¶

16-825 Learning for 3D: Assignment 5

Author: Benny Zhu (haozhenz)
Date: November 2026

Table of Contents¶

  • 1. Classification Model (30 points)
  • 2. Segmentation Model (30 points)
  • 3. Robustness Analysis (20 points)
    • 3.1 Experiment 1: Rotation (10 points)
    • 3.2 Experiment 2: Number of Points (10 points)

1. Classification Model (30 points)¶

python train.py --task cls
python eval_cls.py

Test accuracy: 97.27%

Chair accuracy: 99.84%

Vase accuracy: 92.16%

Lamp accuracy: 92.74%

Correct Predictions¶

Point Cloud
Predicted chair vase lamp
Ground Truth chair vase lamp

Failure Cases¶

Chair Vase Lamp
Point Cloud
Predicted lamp lamp vase
Ground Truth chair vase lamp

All three failures happen when an object's global silhouette is closer to another class than to its own. Chair is almost always right (99.8%) because chairs have a very distinctive structure. Most of the errors are lamp and vase being mistaken for each other, since both are often a round, vertically elongated object on a narrow base.


2. Segmentation Model (30 points)¶

python train.py --task seg
python eval_seg.py --i <object_index>

Test accuracy: 88.35%

Good Predictions¶

Object Ground Truth Prediction Accuracy
#211 78.7%
#249 89.7%
#514 83.7%

Bad Predictions¶

Object Ground Truth Prediction Accuracy
#26 42.0%
#163 32.6%

The model reliably segments the large, geometrically distinct parts: the legs or base (blue), the seat (red), and the backrest (cyan) are correct on all three random chairs, including unusual designs such as the egg chair's pedestal base. Most of the errors come from the armrests (yellow). Armrests sit at the same height as the seat or back and touch both, so the model often assigns them to a neighbor.

These failure modes match how PointNet works. Each point is classified from its own coordinates plus a single global feature, so the predictions mainly follow height and overall position. The model has no way to use local neighborhood geometry to find where one part ends and the next begins. It works well when parts are clearly separated in space, and breaks down for touching parts (armrests), rare classes, and unusual shapes. The two worst objects (#26, #163) are boxy sofa-like chairs where all three problems occur together, and the model essentially splits them horizontally by height.


3. Robustness Analysis (20 points)¶

3.1 Experiment 1: Rotation (10 points)¶

python eval_cls.py --rot_deg <angle>
python eval_seg.py --rot_deg <angle>

Procedure: I rotated every test point cloud around the z-axis by 15°, 30°, 45°, 60°, and 90° and evaluated the trained Q1 and Q2 models on the rotated inputs without retraining. The same 10,000 sampled points were used in every run, so rotation is the only change compared with the Q1/Q2 baselines.

Rotation Angle Classification Accuracy Segmentation Accuracy
0° (Q1 / Q2 baseline) 97.27% 88.35%
15° 93.91% 83.16%
30° 66.53% 70.57%
45° 28.65% 58.90%
60° 23.92% 51.40%
90° 24.24% 40.60%

Classification visualizations:

0° 30° 60° 90°
Pred: chair Pred: chair Pred: vase Pred: vase

Segmentation visualizations:

0° 30° 60° 90°
Ground Truth
Prediction
Accuracy 83.7% 57.5% 38.6% 3.5%

Both models are very sensitive to rotation. Accuracy falls steadily as the angle grows. All training objects are upright in the same canonical pose, and we use no rotation augmentation and no T-Net. As a result, the features PointNet learns are tied to absolute coordinates rather than to the object's shape: the tilted chair in the example is still a chair at 30°, but it is predicted as a vase at 60° and 90°. Segmentation fails in the same way. The per-point predictions still follow the original vertical axis: whatever ends up near the top is labeled back, the middle seat, and the bottom legs, regardless of which part a point actually belongs to. So accuracy on the example chair falls from 83.7% to 3.5% at 90°.

3.2 Experiment 2: Number of Points (10 points)¶

python eval_cls.py --num_points <N>
python eval_seg.py --num_points <N>

Procedure: I evaluated the trained Q1 and Q2 models on test point clouds randomly subsampled to 1,000, 100, and 50 points per object, compared with 10,000 points in the original setting. No retraining was done, and the same subset of point indices was used for every object (and as the labels in segmentation).

Number of Points Classification Accuracy Segmentation Accuracy
10000 (Q1 / Q2 baseline) 97.27% 88.35%
5000 97.27% 88.39%
1000 97.38% 88.23%
500 96.96% 87.74%
100 91.40% 81.71%
50 76.29% 75.02%

Classification visualizations:

10000 1000 100 50
Pred: chair Pred: chair Pred: lamp Pred: lamp

Segmentation visualizations:

10000 1000 100 50
Ground Truth
Prediction
Accuracy 95.44% 98.90% 30.00% 20.00%

PointNet is much more robust to fewer points than to rotation. With only 10% of the points (1,000) accuracy is essentially unchanged, and even with 100 points it stays above 80% for both tasks. This follows from the max-pool design. The global feature is determined by a small set of "critical points" that give the maximum response in each channel, so as long as the subsampled cloud still outlines the overall shape, the feature barely changes. Performance only drops sharply at 50 points, when thin structures such as legs and the backrest are covered by just a few points or disappear. In the classification example, the remaining points of the office chair cluster around the seat and the central pole, and that sparse silhouette looks more like a lamp. In the segmentation example, the model mostly loses the rare, thin parts, and with so few points each mistake costs more accuracy.