Homework 5 Point Cloud Processing¶

1. Classification Model¶

Visaulization

Chair Vase Lamp
123 123 123

Failure cases

Gt Class / Pred Class

Vase / Lamp Lamp / Vase
123 123

All chairs are classified correctly.

2. Segmentation Model¶

Visualization

Pred 123 123 123 123 123
Gt 123 123 123 123 123
Acc 99.55 96.05 90.96 45.75 51.93

Average test accuracy: 90.70

For the failure cases, it seems that the model misclassified the leg as the pad and the back as the leg. This is quite reasonable because the leg is usually located just under the pad and has a thin structure. The distinction largely depends on how we define the chair leg.

3. Robustness Analysis¶

Random rotation¶

123 234

The experiments are done with randomly rotate points with [-degrees, +degrees] in either horizontal or elevation.

The performance drops quite sharply; only rotation within ±10 degrees seems to keep reasonable performance. It is quite surprising that the model is not robust to rotation around the Y-axis (horizontal rotation).

Correct prediction

Chair Vase Lamp
123 123 123

Failure cases

Gt Class / Pred Class

Chair/Lamp Vase / Lamp Lamp / Vase
123 123 123

Segmentation

Pred 123 123
Gt 123 123
Acc 37.46 49.19

Reduce number of points¶

123 234

The experiments were designed with randomly dropped points to test the models' performance.

As the figure shows, the classification performance starts dropping when the number of points is around 100, and for segmentation, it is around 1000 points. The classification between these three classes seems to be easy because we don't need that many points. And the segmentation surprisingly drops slower than the classification, perhaps because it can always guess that the bottom is a chair leg.

Point cloud visualization for n_points = [10000, 2500, 625, 150, 50]

123
123
123
123
123
123