The vase predicted as a chair is a wide, boxy base with a small plant on top. The block-shaped base with a flat bottom looks like a chair seat on a base, so the global shape matches chair features. The vase predicted as a lamp is a tall, thin vase with a flower stem rising from it. The stem and the narrow neck look like a lamp pole, so the model matches it to a lamp. The lamp predicted as a vase has a round body with a short shade on top. The large round body looks like a vase, and the small shade is easy to miss. In all three cases the object has shape features that belong to another class, so PointNet's global max-pooled feature lands closer to the wrong class.
Q2. Segmentation Model
Test accuracy
I used the default parameters. The best test accuracy was 90.1%, at epoch 109.
Segmentation results
3 randomly chosen objects and the 2 worst objects.
Object 308: accuracy 0.9272
ground truthprediction
Object 608: accuracy 0.9874
ground truthprediction
Object 98: accuracy 0.9646
ground truthprediction
Object 351 (bad prediction): accuracy 0.4576
ground truthprediction
Object 163 (bad prediction): accuracy 0.4754
ground truthprediction
The model does well on regular chairs. The parts of these chairs have well-defined boundaries, where surfaces in different plane directions meet at a bend or corner, so the predicted parts match the ground truth almost exactly and the few errors sit right at those boundaries.
The two bad cases fail where the ground-truth boundaries are not marked by a bend in the geometry:
Object 351: the model segments the seat well, because there is a bend between the seat and the surrounding surface. The other boundaries are smooth, with no bend to mark them. Because the geometry is smooth, the model is unable to find the boundaries clearly, unlike in the other examples.
Object 163: the surface flows smoothly from one part to the next, and the ground-truth boundaries are not at plane intersections but inside the continuous surface. The model merges the top part into the neighboring part and mixes up the thin seat layer, placing the splits at the usual positions instead.
In both cases nothing in the geometry marks where one part ends, so the model falls back on the typical layout of a chair.
Q3. Robustness Analysis
Both experiments use the best models from Q1 and Q2.
Experiment 1: rotating the test point clouds
Procedure: I rotate every test point cloud about the z axis by a fixed angle and measure test accuracy on the full test set.
Rotation (deg)
Classification
Segmentation
0 (Q1 / Q2)
0.9780
0.9014
30
0.8248
0.7116
60
0.4984
0.5102
90
0.2949
0.3560
180
0.6086
0.3335
For the visualizations I use the same chair, vase and lamp for classification (all correct at 0°) and test object 0 for segmentation.
The model copes well with fewer points. Classification barely changes even at 100 points, and segmentation holds up until about 500 points.
Segmentation drops faster because it has to label every point, and with fewer points the detail near part boundaries gets thin.
Max pooling only keeps the strongest responses, so the global feature stays similar as long as the points still cover the overall shape.
Q4. Bonus Question - Locality
Model
I implemented DGCNN. Each EdgeConv layer picks the 10 nearest neighbors of every point in feature space, combines each point with its neighbors, runs a shared linear layer with BatchNorm and LeakyReLU, and max-pools over the neighbors. So every point gets some local context, which PointNet does not have.
Classification: 4 EdgeConv layers, a global max pool, then 3 fully connected layers with dropout.
Segmentation: 3 EdgeConv layers, a global max pool, then the global feature is attached to every point before the per-point layers.
I trained with batch size 16, using 5000 points per object for classification and 10000 for segmentation. Classification has wider EdgeConv layers (up to 256 channels, against 64 for segmentation) and ran out of GPU memory at 10000 points, so I used 5000. Everything else is the same as Q1 and Q2.
Test accuracy
Task
PointNet (Q1 / Q2)
DGCNN
Classification
0.9748 (5000 pts)
0.9811 (5000 pts, epoch 97)
Segmentation
0.9014 (10000 pts)
0.9197 (10000 pts, epoch 136)
I visualize the same objects as in Q1 and Q2 and compare the predictions. I check whether DGCNN fixes the objects that failed earlier, and whether it fails on anything new that PointNet got right.
Classification - Visualization
DGCNN gets all 3 Q1 failures right: the vase predicted as a chair, the vase predicted as a lamp and the lamp predicted as a vase.
It also fails on some new objects. Two chairs are predicted as lamps, which PointNet never did, and some vases and lamps get mixed up.
Some objects are still wrong in both models, mostly vases predicted as lamps and lamps predicted as vases.