Assignment 5: Point Cloud Classification and Segmentation
Author: Po-Ting Ko
Andrew ID: potingk
Implementation
Both models are vanilla PointNet without the transformation blocks.
- Backbone. A shared per-point MLP
3 → 64 → 64 → 64 → 128 → 1024(1×1 conv + BN + ReLU), followed by a max-pool over points to get a 1024-d global feature. - Classification head. FC
1024 → 512 → 256 → 3with BN, ReLU and dropout 0.3. - Segmentation head. Each point's 64-d local feature (after the second layer) is concatenated with the global feature, then a per-point MLP
1088 → 512 → 256 → 128 → 6. - Training. Adam (lr 1e-3, cosine decay to 1e-5), batch size 32, 100 epochs, cross-entropy loss. Each training step uses a random subset of 2,500 of the 10,000 points, re-drawn every batch. This is 4–8× faster than using all points and acts as data augmentation. All test numbers below use all 10,000 points.
Q1. Classification Model
Test accuracy: 98.01% (934 / 953)
| Class | Accuracy |
|---|---|
| Chair | 100.00% (617 / 617) |
| Vase | 94.12% (96 / 102) |
| Lamp | 94.44% (221 / 234) |
Random test samples
Failure cases
All 19 errors at 10,000 points are among vases and lamps: 12 lamps → vase, 6 vases → lamp, 1 lamp → chair. The model classifies every chair correctly in this setting. For the chair failure, I therefore use the same model with 1,000 input points (python eval_cls.py --num_points 1000 --i 595). At 1,000 points, #595 is the only misclassified chair.
| Vase → lamp (#651) | Vase → lamp (#673) | Lamp → vase (#726) |
|---|---|---|
![]() |
![]() |
![]() |
| p(lamp) = 1.00 | p(lamp) = 1.00 | p(vase) = 0.99 |
| Lamp → vase (#827) | Lamp → chair (#944) | Chair → lamp (#595, 1,000 points) |
|---|---|---|
![]() |
![]() |
![]() |
| p(vase) = 1.00 | p(chair) = 0.71 | p(lamp) = 0.53, p(chair) = 0.46 |
Interpretation.
- Vase → lamp. Every misclassified vase holds a plant or has a tall thin structure: an orchid (#651), sticks (#670), spire-like shapes (#673, #685). A thin vertical part on top of a base looks like a lamp's pole on its stand.
- Lamp → vase. These lamps have a large, closed, rounded body and no visible thin pole: a ball-shaped base (#726), drum ceiling lamps (#827, #883), a globe wall lamp (#787), a lantern (#859). Their silhouette is the same as a vase's body.
- Lamp → chair (#944). A dome lamp with a wide flat rim. The large flat horizontal surface is a strong chair (seat) cue.
- Chair → lamp (#595). A lounge chair made of one thin curved band. From the side it looks like a gooseneck lamp arm, and it has no seat, back or legs in the usual layout. Even at 10,000 points the model is barely correct (p(chair) = 0.52, p(lamp) = 0.47). With fewer points the balance tips to lamp: for 3 of 5 random subsets at 2,500 points and 4 of 5 at 1,000 points.
Since PointNet sees only one global max-pooled feature, it classifies mainly by coarse shape and proportions. Vases and lamps overlap in exactly these cues, and they are also the two smallest training classes (741 vases and 1,554 lamps vs. 4,489 chairs).
Q2. Segmentation Model
Test accuracy: 90.45% (over all 617 × 10,000 points). The median per-object accuracy is 93.9%.
Colors: cyan = back, red = seat, blue = legs / base, yellow = arms, magenta and white = rare parts (e.g. headrest). The part names are read from the renderings.
| Object | Ground truth | Prediction | Accuracy |
|---|---|---|---|
| #297 | ![]() |
![]() |
99.68% |
| #0 | ![]() |
![]() |
94.52% |
| #134 | ![]() |
![]() |
84.98% |
| #500 | ![]() |
![]() |
74.69% |
| #26 (bad) | ![]() |
![]() |
48.53% |
| #255 (bad) | ![]() |
![]() |
47.41% |
| #351 (bad) | ![]() |
![]() |
45.39% |
Interpretation.
- Standard chairs are nearly perfect. A chair with a separate back, seat and four legs (#297, #0) reaches 95–100%. The remaining errors sit on the part boundaries, for example where the back meets the seat.
- The model relies on height. In the failure cases, it labels most points below the seat as legs and most points above it as back. This breaks for chairs that do not follow that layout:
- Chair #351 is a barrel chair with no legs. The ground truth labels the wrapped sides as arms (yellow), but the model predicts its lower half as legs.
- Chair #255 has a solid box base. The model labels the whole block as seat, while the ground truth calls its lower part the base.
- Chair #500 has a thick seat block, and the model moves the seat/leg boundary upward.
- Rare configurations fail. #26 is a recliner with a footrest, a layout that is rare in training. The model mixes up seat, arms and legs across the whole object.
- Ambiguous labels. For these unusual chairs, even the ground-truth boundary (seat vs. base, arms vs. back) is somewhat arbitrary. Some of the "errors" are reasonable alternative partitions.
Q3. Robustness Analysis
Both experiments use the best Q1 and Q2 checkpoints without retraining. The "0°" and "10,000 points" settings are the Q1 / Q2 results.
Experiment 1: Rotation
Procedure. Rotate every test object about the horizontal x-axis by θ ∈ {0°, 15°, 30°, 45°, 60°, 90°, 120°, 180°}. This tilts the object forward (90° lays it on its back, 180° turns it upside down). The rotation is applied to the input points only; the labels stay the same.
| θ | 0° | 15° | 30° | 45° | 60° | 90° | 120° | 180° |
|---|---|---|---|---|---|---|---|---|
| Cls accuracy | 98.01% | 95.80% | 86.46% | 54.98% | 38.41% | 34.31% | 67.37% | 73.56% |
| Seg accuracy | 90.45% | 83.84% | 72.10% | 63.05% | 54.39% | 27.56% | 26.05% | 33.90% |
Classification samples (rotation about x; predicted class below each).
| Object | 0° | 30° | 60° | 90° |
|---|---|---|---|---|
| Chair #312 | chair |
chair |
lamp ✗ |
lamp ✗ |
| Vase #629 | vase |
vase |
vase |
lamp ✗ |
| Lamp #760 | lamp |
lamp |
lamp |
chair ✗ |
Segmentation samples (object #0; per-object accuracy below each).
Interpretation.
- PointNet is not rotation invariant. Its per-point MLP takes raw xyz coordinates, and all training shapes are upright and aligned. Small tilts (15°) cost little, but accuracy collapses from 30° to 90°.
- Classification. Tilted chairs are mostly predicted as lamps (327 of 617 at 45°). At 90°, accuracy (34%) is below the 65% obtained by always answering "chair". Accuracy recovers to 74% at 180°: an upside-down chair keeps the same height profile mirrored, and 455 of 617 upside-down chairs are still classified as chairs.
- Segmentation fails more, and does not recover at 180°. The model labels a point by where that location would be on an upright chair, not by the part it belongs to. In the 90° sample, the back now lies horizontally at about seat height and is predicted as seat and arms, while the seat, now vertical, is predicted as back. Already at 30–60°, the tilted seat is absorbed into the back and legs. Upside down, the legs are on top, so almost every part is wrong.
- Possible fixes. Rotation augmentation during training, or a learned alignment such as the T-Net we omitted.
Experiment 2: Number of input points
Procedure. Evaluate with N ∈ {10000, 5000, 2500, 1000, 500, 100, 50, 10} points per object. The same random subset of point indices (seed 0) is used for every object; segmentation accuracy is measured on the kept points.
| N | 10,000 | 5,000 | 2,500 | 1,000 | 500 | 100 | 50 | 10 |
|---|---|---|---|---|---|---|---|---|
| Cls accuracy | 98.01% | 98.01% | 97.80% | 97.80% | 97.38% | 93.81% | 83.53% | 28.65% |
| Seg accuracy | 90.45% | 90.53% | 90.60% | 90.36% | 89.37% | 82.30% | 78.60% | 67.57% |
Classification samples (predicted class below each; points are drawn larger at low N).
| Object | 10,000 | 1,000 | 100 | 50 |
|---|---|---|---|---|
| Chair #312 | chair |
chair |
chair |
chair |
| Vase #629 | vase |
vase |
lamp ✗ |
lamp ✗ |
| Lamp #760 | lamp |
lamp |
lamp |
lamp |
Segmentation samples (prediction, per-object accuracy below each).
Interpretation.
- Very robust down to ~500 points. Accuracy changes by less than 1% from 10,000 to 500 points. The global feature is a max over points, and it depends only on a small set of "critical points" that span the shape. A random subset of a few hundred points still covers them well. Training on random 2,500-point subsets also helps.
- Degrades below 100 points. With 50 points, thin parts (legs, lamp poles, plant stems) are hit by only one or two points, so the shape becomes ambiguous. The vase in the samples turns into a lamp at 100 points.
- Segmentation degrades more gracefully. It keeps 68% even at 10 points, while classification drops to 29%. A point's own height is already a strong cue for its part label (see Experiment 1), so per-point labels survive even when the global shape is lost.
Q4. Bonus: Locality (DGCNN)
Model
I implemented DGCNN (Dynamic Graph CNN, Wang et al. 2019). PointNet processes every point independently and only combines them in the final max-pool, so it never sees local structure. DGCNN adds locality with EdgeConv:
- For each point, find its k = 20 nearest neighbours, apply a shared MLP to every edge feature [xj − xi, xi], and max-pool over the neighbours. The relative term xj − xi describes the local shape around the point.
- The graph is dynamic: after the first layer, the kNN graph is rebuilt in the current feature space, so later layers group points that are semantically similar, not only spatially close.
- Classification: 4 EdgeConv layers (64, 64, 128, 256), their outputs concatenated and lifted to 1024-d, then max + average pooling and FC
2048 → 512 → 256 → 3(dropout 0.5). 1.80M parameters (PointNet: 0.81M). - Segmentation: 3 EdgeConv blocks (64, 64, 64); the 192-d multi-scale local features are concatenated with a 1024-d global feature and fed to a per-point MLP
1216 → 256 → 256 → 128 → 6. 0.64M parameters, fewer than PointNet's 0.88M. - No T-Net, same as Q1/Q2. LeakyReLU (0.2) and BN everywhere.
Training. Adam (lr 1e-3, cosine decay), 100 epochs. Classification uses random 1,024-point subsets (batch 32), segmentation random 2,048-point subsets (batch 16). Training ran on an AWS g5.xlarge (A10G 24GB): about 30 s per epoch for each task.
Testing on 10,000 points. The kNN neighbourhood depends on point density: with 10,000 points, the 20 nearest neighbours cover a much smaller region than during training. Feeding all 10,000 points directly drops classification accuracy from 98.7% (at 1,000 points) to 93.9%. So at test time I split each cloud into random disjoint subsets of the training size (1,024 or 2,048 points), predict each subset, and average the class scores (cls) or write each point's scores back (seg). Every one of the 10,000 points still gets a prediction, so the numbers below are directly comparable to Q1 and Q2.
Test accuracy
| Task | PointNet (Q1 / Q2) | DGCNN | Change |
|---|---|---|---|
| Classification | 98.01% (19 errors) | 98.64% (13 errors) | +0.63 |
| Segmentation | 90.45% | 91.89% | +1.44 |
| Class | PointNet | DGCNN |
|---|---|---|
| Chair | 100.00% | 100.00% |
| Vase | 94.12% | 93.14% |
| Lamp | 94.44% | 97.44% |
Both computed with eval_cls.py / eval_seg.py --model dgcnn on all 10,000 points (seed 0).
Classification compared with Q1
DGCNN fixes 11 of PointNet's 19 errors and makes 5 new ones; 8 objects are wrong for both.
- Fixed. Most fixes are lamps that PointNet called vases (#726, #806, #864, #883, #916, #922, #927), plus the dome lamp #944 and three vases holding plants (#667, #670, #714). Local edge features describe surface detail, such as leaves vs. a smooth shade, which a single max-pooled global feature misses.
- Still wrong. Objects whose overall shape is genuinely ambiguous stay wrong: the orchid on a thin stem (#651) and the drum ceiling lamp (#827).
- New errors. DGCNN's new mistakes are vases that look like lamps: a topiary tree on a thin stem (#716) and a stepped dome shaped like a lampshade (#699).
Segmentation compared with Q2
DGCNN is better on 428 of 617 chairs (mean per-object gain +1.4 points).
| Object | Ground truth | Q2 PointNet | DGCNN |
|---|---|---|---|
| #297 | ![]() |
99.68% |
99.48% |
| #0 | ![]() |
94.52% |
95.34% |
| #134 | ![]() |
84.98% |
87.66% |
| #500 | ![]() |
74.69% |
74.48% |
| #26 | ![]() |
48.53% |
59.45% |
| #255 | ![]() |
47.41% |
62.91% |
| #351 | ![]() |
45.39% |
47.70% |
- Better part boundaries. On #134, DGCNN separates the arm (yellow) that PointNet merged into the seat and back. On #255, it starts to label the bottom of the box base as base instead of seat, and on #26 it labels the footrest consistently as seat. These are the cases where height alone is misleading and the local shape (a vertical side panel vs. a horizontal cushion) decides the part.
- Same failure on #351. Both models still call the lower half of the barrel chair "legs". With no example of this layout, the global position prior wins.
- Noisier labels in ambiguous regions. DGCNN's labels are speckled where parts meet (e.g. #255). Neighbouring points can fall into different test-time subsets and are predicted separately, so uncertain regions mix labels.
Robustness compared with Q3
The same two experiments as in Q3, for both models. For DGCNN, inputs larger than the training size use the subset strategy above; smaller inputs are fed directly.
| Setting | Cls PointNet | Cls DGCNN | Seg PointNet | Seg DGCNN |
|---|---|---|---|---|
| Original | 98.01% | 98.64% | 90.45% | 91.84% |
| Rotate 15° | 95.80% | 97.90% | 83.84% | 89.93% |
| Rotate 30° | 86.46% | 80.69% | 72.10% | 77.87% |
| Rotate 60° | 38.41% | 35.47% | 54.39% | 35.73% |
| Rotate 90° | 34.31% | 34.63% | 27.56% | 17.74% |
| 500 points | 97.38% | 97.27% | 89.37% | 87.34% |
| 100 points | 93.81% | 65.37% | 82.30% | 59.89% |
| 50 points | 83.53% | 36.62% | 78.60% | 42.92% |
The "Original" DGCNN numbers come from the sweep script and differ from the table above by ≤ 0.05 points because the test-time subsets are drawn at random.
- Small rotations: DGCNN is more robust. At 15° DGCNN loses 0.7 (cls) and 1.9 (seg) points, PointNet 2.2 and 6.6. The edge feature xj − xi depends on the local shape, which changes little under a small tilt.
- Large rotations: neither model is rotation invariant. Both still take absolute coordinates as input, and DGCNN's segmentation is even worse at 60–120°. Locality alone does not give rotation invariance; that would need rotation augmentation or rotation-invariant features.
- Sparse inputs: locality is a weakness. Below 500 points, DGCNN collapses much faster than PointNet (65% vs. 94% classification at 100 points). With 100 points, 20 nearest neighbours already cover a fifth of the object, so the "local" features look nothing like those seen in training. PointNet's per-point features do not depend on neighbours, so a few hundred points are enough for it. The same density sensitivity is why DGCNN needs the subset strategy for 10,000-point inputs.













































































