Assignment 5: Point clouds

Andrew ID: lnayak

1. Classification

The PointNet classifier uses shared pointwise layers (64, 64, 64, 128, 1024), max pooling, and a 512/256 classifier. It has no transformation blocks. Softmax converts its class scores into probabilities; the largest score gives the predicted class.

All four models train for 100 epochs with Adam, batch size 64, learning rate 0.001 decaying to 0.00001, weight decay 0.0001 and seed 825. Each training pass samples 1,024 points without replacement and applies random scaling and translation. There is no rotation augmentation. A fixed 10% of the training data is held out for validation (stratified for classification). The checkpoint with the highest validation accuracy is used for testing.

Test accuracy: 97.27% on all 953 test objects using 10,000 points per object.

True classTest objectsAccuracy
Chair617100.00%
Vase10283.33%
Lamp23496.15%

Random test examples

Six test objects sampled without replacement with seed 825.

Object 183. Ground truth: chair. Prediction: chair.
Object 183. Ground truth: chair. Prediction: chair.
Object 359. Ground truth: chair. Prediction: chair.
Object 359. Ground truth: chair. Prediction: chair.
Object 900. Ground truth: lamp. Prediction: lamp.
Object 900. Ground truth: lamp. Prediction: lamp.
Object 486. Ground truth: chair. Prediction: chair.
Object 486. Ground truth: chair. Prediction: chair.
Object 90. Ground truth: chair. Prediction: chair.
Object 90. Ground truth: chair. Prediction: chair.
Object 156. Ground truth: chair. Prediction: chair.
Object 156. Ground truth: chair. Prediction: chair.

Failure examples

There are no misclassified true chairs in the test set. The chair example is therefore a false positive: a lamp predicted as a chair. For vase and lamp, I show the first misclassified object of that true class.

Chair false positive. Object 758: lamp predicted as chair.
Chair false positive. Object 758: lamp predicted as chair.

This is a wall-mounted lamp with a curved bracket and a cage-like body. Its unusual frame is a plausible source of confusion with chair geometry. This is a chair false positive; none of the actual chairs are misclassified.

Vase false negative. Object 619: vase predicted as lamp.
Vase false negative. Object 619: vase predicted as lamp.

The vase contains a tall flower stem above a narrow container. The stem and flower create a silhouette similar to a lamp stand and shade. The input contains only coordinates, so the classifier cannot use color or material to separate them.

Lamp false negative. Object 787: lamp predicted as vase.
Lamp false negative. Object 787: lamp predicted as vase.

The lamp has two rounded sections and no obvious thin stand. Its bulbous shape resembles a decorative vase. The model predicts vase even though the dataset labels it as a lamp.

2. Segmentation

The segmentation model concatenates a 64-channel feature at each point with the pooled 1,024-channel global feature. Shared layers of width 512, 256 and 128 predict six part scores at every point.

Test point accuracy: 90.13% over all 617 chairs and 6,170,000 points. Every point is scored, including the small part classes.

Part colors: 0 gray, 1 purple, 2 cyan, 3 yellow, 4 blue, 5 red. Three chairs are randomly selected with seed 825; the other two are the lowest-accuracy predictions. Ground truth and predictions use the same points, scale and rotating camera.

Object 271. Accuracy: 92.73%.
Object 271. Accuracy: 92.73%.

The broad back, seat and four legs are mostly assigned the correct labels. The remaining mistakes occur around transitions between these parts and some narrow structures.

Object 514. Accuracy: 85.73%.
Object 514. Accuracy: 85.73%.

The model recovers the main seat and back, but the yellow side regions spread into nearby parts. Several boundaries are less precise than in the ground truth, which lowers accuracy despite the recognizable overall segmentation.

Object 436. Accuracy: 95.76%.
Object 436. Accuracy: 95.76%.

The rectangular back, seat and slender legs are well separated. Most errors are small patches near part boundaries rather than large mislabeled surfaces.

Object 163. Accuracy: 37.76%. One of the two worst predictions.
Object 163. Accuracy: 37.76%. One of the two worst predictions.

This unusual chair has a purple upper part and a separate-looking gray region in front of the seat. The prediction largely loses both regions and replaces them with the more common cyan and red labels. Those large errors explain the low point accuracy.

Object 605. Accuracy: 40.94%. One of the two worst predictions.
Object 605. Accuracy: 40.94%. One of the two worst predictions.

Unlike the simple four-legged examples, this chair has a large triangular support below the seat. The model extends yellow arm labels into the blue support region and also confuses parts around the seat. Local coordinates alone are less informative for this unusual structure.

3. Robustness

Rotation

I rotate every test cloud about its Y axis by 0, 15, 30, 45, 90 and 180 degrees. Labels and point count stay unchanged, and neither model is retrained. The same 953 classification objects and 617 segmentation objects are evaluated at each angle.

Y rotation (degrees)Classification accuracySegmentation accuracy
097.27%90.13%
1596.43%86.78%
3091.82%78.81%
4563.69%70.05%
9052.36%54.07%
18032.74%46.14%
Test accuracy under rotation.
Test accuracy under rotation.

At 90 degrees, classification changes from 97.27% to 52.36%, and segmentation from 90.13% to 54.07%. Max pooling makes PointNet insensitive to point ordering, but does not make it rotation invariant. Rotating a chair changes the coordinate patterns associated with its parts, and training did not include these rotations.

cls, object 183: original input and 90-degree rotation. True class: chair; predictions: chair, vase.
cls, object 183: original input and 90-degree rotation. True class: chair; predictions: chair, vase.
cls, object 359: original input and 90-degree rotation. True class: chair; predictions: chair, lamp.
cls, object 359: original input and 90-degree rotation. True class: chair; predictions: chair, lamp.
seg, object 271: original input and 90-degree rotation.
seg, object 271: original input and 90-degree rotation.
seg, object 514: original input and 90-degree rotation.
seg, object 514: original input and 90-degree rotation.

Number of points

I sample 2,048, 1,024, 512, 256 and 128 points without replacement from each test object, with three seeds (825, 826, 827). Subsets are nested within each seed, and segmentation labels use exactly the same indices. The table reports mean accuracy and standard deviation across the three samplings. The 10,000-point row is the original full-cloud result. Models are not retrained.

PointsClassification accuracySegmentation accuracy
10,00097.27%90.13%
2,04897.06% ± 0.09%90.51% ± 0.02%
1,02496.92% ± 0.13%90.71% ± 0.04%
51296.68% ± 0.18%90.44% ± 0.07%
25696.08% ± 0.26%88.47% ± 0.13%
12894.51% ± 0.26%84.81% ± 0.28%
Test accuracy versus point count; error bars show sampling standard deviation.
Test accuracy versus point count; error bars show sampling standard deviation.

Classification stays close to the full-cloud result through 512 points, then drops more clearly at 128. Segmentation is slightly better around the 1,024-point training density than at 10,000 points, but falls at 256 and 128 points. Sparse inputs can miss thin supports and part boundaries; they also change the features available to global max pooling.

Segmentation is scored on the sampled points, so these measurements reflect both less geometric context and a different set of evaluated points. Small parts may be absent in a sparse sample. The figures use seed 825 and the same objects as the rotation experiment.

cls, object 183: 10,000 points compared with 128 points. True class: chair; predictions: chair, chair.
cls, object 183: 10,000 points compared with 128 points. True class: chair; predictions: chair, chair.
cls, object 359: 10,000 points compared with 128 points. True class: chair; predictions: chair, chair.
cls, object 359: 10,000 points compared with 128 points. True class: chair; predictions: chair, chair.
seg, object 271: 10,000 points compared with 128 points.
seg, object 271: 10,000 points compared with 128 points.
seg, object 514: 10,000 points compared with 128 points.
seg, object 514: 10,000 points compared with 128 points.

4. Bonus: locality

I use a DGCNN-style network with three EdgeConv blocks of width 64, 64 and 128. Each block finds 20 nearest neighbors and applies a shared network to the center feature and the neighbor-minus-center feature, then max-pools over neighbors. Neighbors are recomputed in the current feature space at every block. Concatenated local features are projected to 512 channels. Classification uses global max and mean pooling; segmentation combines local features with the pooled global feature.

Reference: Dynamic Graph CNN for Learning on Point Clouds.

Training split, augmentation, point count, epochs, optimizer and checkpoint selection match PointNet. This is a smaller DGCNN-style implementation, not a reproduction of the full paper model. For a direct comparison, both models receive the same 1,024 points from every test object (seed 825). The Q1/Q2 full-cloud results are also listed separately.

TaskPointNet, 10,000 pointsPointNet, 1,024 pointsEdgeConv, 1,024 pointsDifference at 1,024 (pp)
Classification97.27%96.96%98.11%+1.15
Segmentation90.13%90.71%91.43%+0.72

EdgeConv improves overall test accuracy in both tasks at the matched point count. Neighbor-relative features provide local shape cues that PointNet has to infer from isolated point features and a global summary. This does not improve every object: the three randomly selected segmentation examples below all score lower with EdgeConv. I also show the same two difficult chairs and the same classification failure cases from earlier sections, so improvements and remaining errors can be compared directly.

cls, object 183: PointNet and EdgeConv on identical input points.
cls, object 183: PointNet and EdgeConv on identical input points.
cls, object 359: PointNet and EdgeConv on identical input points.
cls, object 359: PointNet and EdgeConv on identical input points.
cls, object 900: PointNet and EdgeConv on identical input points.
cls, object 900: PointNet and EdgeConv on identical input points.
cls, object 758: PointNet and EdgeConv on identical input points.
cls, object 758: PointNet and EdgeConv on identical input points.
cls, object 619: PointNet and EdgeConv on identical input points.
cls, object 619: PointNet and EdgeConv on identical input points.
cls, object 787: PointNet and EdgeConv on identical input points.
cls, object 787: PointNet and EdgeConv on identical input points.
seg, object 271: PointNet and EdgeConv on identical input points.
seg, object 271: PointNet and EdgeConv on identical input points.
seg, object 514: PointNet and EdgeConv on identical input points.
seg, object 514: PointNet and EdgeConv on identical input points.
seg, object 436: PointNet and EdgeConv on identical input points.
seg, object 436: PointNet and EdgeConv on identical input points.
seg, object 163: PointNet and EdgeConv on identical input points.
seg, object 163: PointNet and EdgeConv on identical input points.
seg, object 605: PointNet and EdgeConv on identical input points.
seg, object 605: PointNet and EdgeConv on identical input points.

These are single training runs. Neighborhoods add local geometric information, but they also change parameter count and computation, so the comparison does not isolate locality from every other model difference.

Training and validation accuracy for all four models.
Training and validation accuracy for all four models.