16-825 · Assignment 5 · Jinxi He

PointNet on point clouds

Q1. Point cloud classification

Test accuracy

PointNet classified 910 of 953 objects correctly (95.49%) using all 10,000 points per object. It recognized every chair. Vases were the hardest class, with 23 of 102 mistaken for lamps.

Test objects by true and predicted class
Ground truthPred. chairPred. vasePred. lamp
Chair61700
Vase37623
Lamp413217

Predictions for all 953 test objects (CSV)

Model and training

A shared MLP maps each XYZ point to 64, 128 and then 256 features. Max pooling across points produces one feature vector per object. A 128-unit hidden layer with dropout (0.3) maps that vector to three class scores.

I trained on 6,106 objects with 1,024 randomly sampled points per object, using Adam, batch size 32 and a cosine learning-rate schedule starting at 0.001. I held out 678 training objects for validation. Epoch 20 had the highest validation accuracy (96.61%) over 40 epochs.

Random test objects

Six random point clouds. Objects 621, 412, 818, 417 and 84 are correct. Lamp 734 is predicted as a chair.
Six test objects sampled with seed 42. Each title gives the object index, true class and predicted class.

Where the classifier failed

Vase 718 is predicted as a lamp, and lamp 916 as a vase.
Errors from the epoch-20 model used for the test accuracy above.

Vase #718. The wide rim and shallow body give this vase the outline of a lampshade. The open top is visible, but the overall shape lacks the narrow neck seen on many vases.

Lamp #916. A thin stem supports a much broader top. Its centered base and flared profile resemble a pedestal vase. The overhanging shade is the clearest visual clue to its true class.

One test error per class at epoch 10

The validation-selected model got all 617 chairs right. To show a genuine chair error, I also evaluated the epoch-10 checkpoint on the same test set. It scored 901/953 (94.54%). All three predictions below come from that checkpoint.

At epoch 10, chair 543 and vase 714 are predicted as lamps. Lamp 914 is predicted as a vase.
Left to right: a chair with a tall slatted back, a vase on a stand, and a lamp with a bulbous body.

Chair #543. Its back extends far above a small seat and short front legs. The narrow upright outline resembles a floor lamp. This was the only chair error at epoch 10, and the model got it right at epoch 20.

Vase #714. A slim pedestal and curled feet support the bowl, giving it a floor-lamp profile. The foliage above the rim distinguishes it from a lampshade. Both checkpoints predicted lamp.

Lamp #914. The rounded body tapers to a narrow neck, much like a vase. The stepped base is visible, but there is no broad shade in this point cloud. The epoch-20 prediction was correct.

Epoch-10 test predictions (CSV)

Q2. Chair part segmentation

PointNet correctly labeled 5,544,153 of 6,170,000 test points (89.86%) across 617 chairs. The median chair scored 92.97%.

Each point gets a 64-dimensional local feature. Two more shared layers and max pooling produce a 256-dimensional feature for the whole chair. I concatenate the two at each point, then apply shared layers of width 128 and 64 to predict six part scores.

I trained for 40 epochs on 4,041 chairs with 1,024 sampled points per chair and selected the checkpoint on 448 held-out chairs. Epoch 38 had the highest validation point accuracy (89.86%). Test accuracy counts correct labels across all 10,000 points on each of the 617 test chairs. Accuracy for every test chair (CSV)

Three chairs across the score distribution

These chairs sit at the 25th, 50th and 75th percentiles of per-chair accuracy. The four ground-truth parts visible in these five examples use the same colors throughout Q2 and Q3.

Chair #256 · 86.57%

Ground truthGround-truth part labels rotating on chair 256
PredictionPredicted part labels rotating on chair 256
The padded sides meet the seat along a broad seam. The prediction mixes pink arm points with the orange seat in this region. Only 56 of the 168 points on the small green feet keep the correct label.

Chair #384 · 92.97%

Ground truthGround-truth part labels rotating on chair 384
PredictionPredicted part labels rotating on chair 384
The blue back, orange seat and four green legs are easy to distinguish in both views. The largest exchange is between seat and support. The model assigns 237 seat points to the supports and 125 support points to the seat.

Chair #298 · 96.43%

Ground truthGround-truth part labels rotating on chair 298
PredictionPredicted part labels rotating on chair 298
The model follows the curved blue back and gets all 5,610 orange seat points right. Of the 357 errors, 324 are green support points absorbed into the seat.

Two lowest-scoring chairs

Chair #351 · 46.41%

Ground truthGround-truth part labels rotating on chair 351
PredictionPredicted part labels rotating on chair 351
The arms and back form a continuous curved shell that reaches the floor. The model colors much of the lower shell green, assigning 3,300 points to the support class even though the ground truth has no points of that class on this chair.

Chair #272 · 46.50%

Ground truthGround-truth part labels rotating on chair 272
PredictionPredicted part labels rotating on chair 272
The opening in the blue back survives in the prediction. The largest error is on the broad panels below the seat. The model assigns 5,235 of the 7,318 green support points to the orange seat, extending the seat down the sides.

On chairs with distinct seats, arms and legs, errors tend to cluster where two parts meet. On the shell chair (#351) and the chair with a panel base (#272), a whole surface takes the wrong label. These larger errors pull the overall point accuracy below the median chair’s score.

Q3. Robustness to rotation and point count

I kept the Q1 and Q2 checkpoints fixed and evaluated every condition on the same 953 classification objects and 617 segmentation chairs. The full, unrotated clouds reproduced the Q1 total of 910 correct objects and the Q2 total of 5,544,153 correctly labeled points.

Rotating the input

I rotated every XYZ coordinate about the vertical Y axis through the origin, using the same angle for all objects. Each cloud kept all 10,000 points and their original labels.

Test accuracy by yaw angle
YawClassificationSegmentation
0°95.49%89.86%
30°93.18%79.32%
60°83.95%61.60%
90°77.33%56.70%

A quarter turn reduced accuracy by about 18.2 percentage points for classification and 33.2 for segmentation. The vertical coordinate stayed fixed throughout this sweep. The drop suggests that both models use the training set’s front-to-back orientation, especially when assigning part labels.

Chair 412 and vase 621 keep their labels after a quarter turn. Lamp 818 changes to chair.
The same three objects from Q1, one per class. Lamp #818 changes from lamp to chair when the curved stem faces the other way. Each caption gives the true and predicted class.
Three chairs before and after rotation, with ground truth for the rotated inputs. On chair 384 the predicted back changes from blue to pink.
For chair #384, rotation turns much of the blue back into a predicted pink arm. Its point accuracy falls from 93.0% to 65.0%. The right column shows the original part labels on the rotated geometry.

Reducing the number of points

For each unrotated object, I shuffled its point indices without replacement and kept the first 5,000, 1,000 or 250. The smaller samples are nested within the larger ones. I repeated this for seeds 42, 43 and 44. Classification scores one label per object. Segmentation scores only the retained points against their matching labels.

Mean test accuracy, with the range across the three draws in parentheses
PointsClassificationSegmentation
10,00095.49%89.86%
5,00095.77% (95.70–95.91)89.90% (89.89–89.92)
1,00095.24% (95.07–95.38)90.02% (89.95–90.08)
25094.72% (94.23–95.07)88.20% (88.10–88.34)

At 1,000 points, both scores stayed within a quarter of a percentage point of their full-cloud baselines. Both models saw 1,024 randomly sampled points per object during training. Max pooling takes the strongest response in each feature channel, so removing points can leave the global feature nearly unchanged if nearby points still produce similar responses. This may explain why even at 250 points the average losses were only about 0.8 percentage points for classification and 1.7 for segmentation.

At 250 points chair 412 and lamp 818 remain correct. Vase 621 changes to lamp.
At 250 points, the curved stem and shade of lamp #818 are still visible and the prediction stays correct. Vase #621 changes to lamp in this draw (seed 42).
Segmentations and ground truth at 250 points. Chair 256 drops to 78.4% while chair 298 retains 96.8% accuracy.
Chair #256 drops from 86.6% on the full cloud to 78.4% on the retained points. The simpler chair #298 still scores 96.8%. Both sparse columns use exactly the same 250 points (seed 42).

Correct counts and denominators for every run (CSV)