The model classifies all 617 test chairs correctly, so there is no chair failure on the original test set. As a substitute, this is a chair that fails after rotating it by 45° about the y axis (from Q3).
The rotated chair fails because the model was trained without rotation augmentation.
The vase is a low, wide object with a flat round base and no tall neck, so it looks unlike a typical vase.
The lamp looks like a vase.
Q2. Segmentation Model (30 points)
PointNet segmentation network: a per-point MLP gives local features, an MLP with max-pooling gives a global feature, and the two are concatenated for every point and passed through a per-point MLP to predict 6 part labels. Trained with Adam (lr 1e-3, batch size 16) for 100 epochs on 10,000 points per object.
Test accuracy: 89.78%
Segmentation results
Object
Ground Truth
Prediction
Accuracy
#397
99.71%
#471
99.45%
#562
99.43%
#235 (bad)
38.89%
#351 (bad)
44.54%
Interpretation:
The good results are normal chairs with simple parts.
The bad results are not normal chairs and have no long legs, so the parts are complicated.
Q3. Robustness Analysis (20 points)
Both experiments use the models from Q1/Q2 (baselines: 98.01% cls, 89.78% seg).
Experiment 1: Rotating the input point cloud
Each test point cloud (all 10,000 points) is rotated by 0–180° about the x, y or z axis before it goes into the network (--rotation_axis, --rotation_angle). The y axis is the vertical axis of the objects.
Seg: GT of object #397Seg: prediction, 0° (99.7%)Seg: prediction, y 90° (79.7%)
Interpretation: accuracy goes down as the angle grows. Classification is robust to rotation about the vertical y axis and not as that robust about z. Under y rotation the vases and lamps stay at about 90%, while chairs prediction accuracy goes down from 100% to 2.3% at 180° (mostly predicted as lamp); a possible reason is that vases and lamps may be symmetric around the vertical axis. It shows that PointNet only works in the orientation it was trained on.
Experiment 2: Varying the number of input points
A random subset of the 10,000 points is used (same subset for all objects, as --num_points). Results are means over 5 random subsets.
Interpretation: both models are robust to fewer points: with 1000 points (10%) classification only drops to 97.3% and segmentation stays at 89.7%. Accuracy falls clearly only below about 500 points; at 50 points classification is 87.3% and segmentation 76.7%. Segmentation degrades faster, probably because every point must be labelled and small parts disappear in a sparse cloud (not verified).
Q4. Bonus: Locality (20 points)
I implemented DGCNN, each EdgeConv layer finds the k = 20 nearest neighbours of every point, applies a shared MLP to the edge features, and max-pools over the neighbours, so each point gets local context. The neighbour graph is recomputed in feature space at every layer. Both models are trained and tested with 1024 points (Adam, cosine learning-rate decay, 40 epochs for cls, 60 for seg).
Discussion: DGCNN is better than PointNet in both tasks at the same number of points (cls 97.25% → 98.41%, seg 89.68% → 91.28%), even compared with PointNet using all 10,000 points. In segmentation it helps most on bulky objects.