16-825 Learning for 3D Vision

Point Cloud Processing

Q1 · Classification

Test accuracy 97.5866%

Random test point clouds

Point cloud of a chair with a curved back and thin legs
Object 487 · Ground truth: chair · Prediction: chair
Point cloud of an armchair
Object 450 · Ground truth: chair · Prediction: chair
Point cloud of a lamp with a rounded shade
Object 719 · Ground truth: lamp · Prediction: lamp

Misclassified point clouds

Point cloud of a chair with a very tall, slatted back
Object 543 · Chair predicted as lamp

The tall, narrow chair back may look like a floor lamp.

Point cloud of a cylindrical vase with a flared rim
Object 618 · Vase predicted as lamp

The wide rim may look like a lampshade.

Point cloud of a lamp with a faceted upper body and a small stand
Object 750 · Lamp predicted as vase

The lamp’s large, angular top may look like a decorative vase.

Q2 · Segmentation

Test accuracy 89.5962%

Object Ground truth Prediction Point accuracy
0 Ground-truth segmentation for chair 0 Predicted segmentation for chair 0 94.0700%
1 Ground-truth segmentation for chair 1 Predicted segmentation for chair 1 98.4100%
2 Ground-truth segmentation for chair 2 Predicted segmentation for chair 2 87.3400%
Failure 351 Ground-truth segmentation for chair 351 Predicted segmentation for chair 351 42.0700%
Failure 26 Ground-truth segmentation for chair 26 Predicted segmentation for chair 26 46.1200%

The model separated the seats, backs, and supports well on chairs 0 and 1. On chair 2, most mistakes appear near the seat and armrest boundaries. For chair 351, the model labeled much of the lower shell as legs. For chair 26, it labeled the headrest as part of the back and much of the armrest as seat.

Q3 · Robustness

Experiment 1 · Rotation

We rotate each test point cloud by 30°, 60°, and 90° around the vertical Y axis and evaluate the trained Q1 and Q2 models.

Experiment 2 · Point count

We evaluate the same models on 2,500, 1,000, and 250 points per object. We sample without replacement using a fixed random seed. Each smaller set is a subset of the larger sets, and segmentation is scored on the retained points.

Condition Classification accuracy Segmentation accuracy
Q1 / Q2 baseline (10,000 points)97.5866%89.5962%
30° rotation91.3956%77.0828%
60° rotation76.7051%56.3565%
90° rotation62.6443%47.5078%
2,500 points97.6915%89.4756%
1,000 points97.0619%89.1066%
250 points96.5373%85.2947%

Rotation results

Accuracy fell as the angle increased. At 90°, classification dropped by 34.94 percentage points and segmentation dropped by 42.09. At 60°, the classifier mistook chair 487 for a vase. At 90°, the segmentation model mislabeled most of chair 0's back.

Classification predictions for the same three objects at each rotation angle
Classification under rotation
Ground truth and part predictions for three chairs at each rotation angle
Segmentation under rotation

Point-count results

Both tasks stayed close to baseline at 1,000 points. At 250 points, classification dropped by 1.05 percentage points and segmentation dropped by 4.30. The three illustrated objects kept their predicted classes, but chair 1's segmentation accuracy fell from 98.4% to 91.6%.

Classification predictions for the same three objects as point count decreases
Classification with fewer points
Ground truth and part predictions for three chairs as point count decreases
Segmentation with fewer points

Q4 · Locality (bonus)

We use a compact DGCNN with two EdgeConv layers and 16 neighbors per point. The first layer finds neighbors in XYZ space. The second rebuilds the graph in the learned feature space. Each layer uses a shared MLP and max pooling over the neighbors. Classification uses a global descriptor, and segmentation combines that descriptor with each point's local features.

We select each DGCNN checkpoint by validation accuracy and compare it with the original PointNet on identical sampled test points.

ModelClassification accuracySegmentation accuracy
PointNet (Q1/Q2, 10,000 points)97.5866%89.5962%
PointNet (1,024 points)97.1668%88.9492%
DGCNN (1,024 points)96.4323%90.3262%

On the same 1,024 points, DGCNN scored 0.73 percentage points lower on classification and 1.38 points higher on segmentation. Both models correctly classified all three objects shown.

PointNet and DGCNN classifications for the same chair, vase, and lamp
Classification comparison on identical test points

On chair 26, DGCNN improved point accuracy from 51.2% to 59.6%. Both models still mislabeled its headrest as part of the back.

Ground truth, PointNet segmentation, and DGCNN segmentation for three chairs
Segmentation comparison on identical test points