Assignment 5: Point Clouds¶
Name: Chenzhe Zhang
Course: 16-825 Learning for 3D Vision
1. Classification¶
I implemented a PointNet classifier with shared pointwise layers, global max pooling, and a fully connected classification head. The model was trained for 250 epochs using Adam with a learning rate of 0.001, a batch size of 16, and 10,000 points per object.
Test accuracy: 97.17% (926 / 953 objects). All results and visualizations below use model_epoch_220.pt, which provides a failure example for each class.
Random test examples¶
I randomly sampled two test objects from each class using seed 825. Ground-truth and predicted classes are shown above each point cloud. The sampled lamp #870 is misclassified as a vase.

Failure cases¶
| Chair → Lamp | Vase → Lamp | Lamp → Vase |
|---|---|---|
![]() |
![]() |
![]() |
- Chair #406 → Lamp: The seat, back, and legs are visible. A possible reason for the error is that the model does not explicitly encode how these parts connect, making it harder to recognize their arrangement.
- Vase #707 → Lamp: The tall, narrow body and broad base give this vase a lamp-like silhouette. These geometric similarities may explain why the model predicts Lamp.
- Lamp #914 → Vase: The rounded body and narrow neck resemble a decorative vase. A model using only point coordinates has limited cues about the object's function, which may contribute to this error.
2. Segmentation¶
I implemented a PointNet segmentation model that concatenates per-point features with a shared global feature, then predicts one of six labels for each point. Training used Adam for 250 epochs, with a learning rate of 0.001, a batch size of 16, and 10,000 points per object.
Test point accuracy: 90.46% (5,581,180 / 6,170,000 points). Evaluation covers all 617 test chairs. All metrics and visualizations use the same segmentation checkpoint: best_model.pt, saved at epoch 247.
Segmentation examples¶
Ground truth is shown on the left and prediction on the right, using the same camera views and label colors. The first three examples are mostly correct; the last two show poor predictions.
| Test chair | Point accuracy | Ground truth / Prediction |
|---|---|---|
| #241 | 98.10% | ![]() |
| #56 | 95.41% | ![]() |
| #315 | 96.10% | ![]() |
| #255 | 54.68% | ![]() |
| #26 | 53.60% | ![]() |
Interpretation¶
- Chair #241: The seat, back, and legs are clearly separated, consistent with the high point accuracy.
- Chair #56: Most parts are segmented correctly, but some armrest points are confused with the back.
- Chair #315: The model preserves the major parts of the office chair, including its star-shaped base.
- Chair #255 (poor prediction): The solid supporting base is mostly assigned the seat label. Its broad, connected surface resembles a continuation of the seat, which may explain the confusion.
- Chair #26 (poor prediction): The headrest is merged into the back, and parts of the seat are confused with the armrests. The headrest label accounts for only about 0.13% of training points, so class imbalance may contribute to this error.
3. Robustness Analysis¶
I used the classification checkpoint from epoch 220 and the segmentation checkpoint from epoch 247. The model weights were fixed in all tests, without further training. I tested all 953 objects for classification and all 617 chairs for segmentation. The figures use examples from Q1 and Q2.
3.1 Input Rotation¶
I rotated each test point cloud around the x axis through the origin. The angles were 0°, 15°, 30°, 45°, 60°, and 90°. Each cloud still has 10,000 points, and all labels stay the same.
| Rotation (degrees) | Classification accuracy (%) | Segmentation point accuracy (%) |
|---|---|---|
| 0 | 97.17 | 90.46 |
| 15 | 93.39 | 83.45 |
| 30 | 82.48 | 71.64 |
| 45 | 57.40 | 62.26 |
| 60 | 36.31 | 44.49 |
| 90 | 24.87 | 25.72 |

The figures below compare the original inputs with the inputs rotated by 90°.
Classification: original vs. rotated

After rotation, chair #119 is predicted as a vase, and lamp #798 is predicted as a chair. Vase #626 is still classified correctly.
Segmentation: original GT/prediction vs. rotated GT/prediction

The four columns show the original ground truth, original prediction, rotated ground truth, and rotated prediction. The same label colors are used in all panels. After rotation, many points on the seats, backs, and legs receive wrong labels.
At 90°, classification accuracy drops by 72.30 percentage points, and segmentation accuracy drops by 64.73 percentage points. Both models are sensitive to rotation. Global max pooling makes the global feature independent of point order, but it does not make the model rotation invariant. Rotating the cloud changes its coordinates, which can change the features and predictions.
3.2 Number of Input Points¶
I sampled points uniformly at random without replacement for each object. I used three random seeds: 825, 826, and 827. For each object and seed, the smaller point sets are part of the larger ones.
The table shows the mean accuracy ± sample standard deviation over the three seeds. The 10,000-point row is the full-input baseline. For segmentation, I measured accuracy only on the selected points.
| Input points | Classification accuracy (%) | Segmentation point accuracy (%) |
|---|---|---|
| 10,000 | 97.17 | 90.46 |
| 5,000 | 97.24 ± 0.06 | 90.42 ± 0.02 |
| 2,000 | 97.10 ± 0.12 | 90.17 ± 0.04 |
| 1,000 | 96.61 ± 0.22 | 89.62 ± 0.03 |
| 500 | 96.29 ± 0.42 | 88.23 ± 0.16 |
| 100 | 94.61 ± 0.53 | 81.45 ± 0.28 |

The dashed line shows a control result for segmentation. The model uses all 10,000 points to make predictions, but accuracy is measured only on the same sampled points. This helps check whether the accuracy drop comes from using fewer input points.
The figures below use 100 points and seed 825.
Classification: 10,000 vs. 100 points

All three examples are still classified correctly with 100 points. Although many surface details are missing, the overall shapes are still recognizable.
Segmentation: dense GT/prediction vs. sparse GT/prediction

The four columns show the full-input ground truth and prediction, followed by the sampled ground truth and prediction. With 100 points, the accuracies of chairs #241, #56, and #315 are 97%, 86%, and 93%, respectively.
With 100 points, the mean accuracy is 94.61% for classification and 81.45% for segmentation. The mean control accuracy is 90.48%. The control stays close to the full-input baseline, suggesting that the segmentation drop mainly comes from using fewer points as input.
In these tests, segmentation is more affected by fewer input points than classification. The overall shape can still support object classification, while segmentation needs more details to separate parts. Sampling can also change the global max-pooled feature. Small accuracy changes at larger point counts can come from random sampling.







