ASSIGNMENT 5¶

Andrew ID : kgaddoba

Test Accuracy : 0.9832

Success :

Failure :

Interpretation :

For the misclassified chair, the model predicted it as a lamp. The chair’s 3D structure in this sample is less distinct than in correctly classified examples, making it harder for the model to recognize typical chair features like the seat and legs.
In the vase misclassification, the model labeled it as a lamp. The point cloud was sparsely sampled due to computational constraints, resulting in missing regions that likely prevented the model from capturing the vase’s full shape.
For the lamp incorrectly predicted as a chair, the object is unusually thin and elongated. Unlike other lamp examples with clear defining features, this shape lacks strong global cues, causing the model to confuse it with the chair class.

Test Accuracy : 0.8986

Success :

Failure :

Interpretation :

In this poor segmentation example, the model struggles with neighboring regions, mislabeling the armrest and side panels. While the backrest and seat are partially identified correctly, noisy and inconsistent boundaries reduce the per-object accuracy.
Another low-accuracy case shows that the model identifies major segments but fails to clearly separate side and back panels. Label spillover occurs across adjacent regions, reflecting the difficulty in handling geometrically similar parts and abrupt spatial transitions.
These observations indicate that while the model can capture the overall shape and main regions of the chair, fine-grained distinctions between adjacent components remain challenging, resulting in reduced accuracy for certain object parts.

EXPERIMNET 1 : ROTATION

a. Procedure :

Applied rotations to all test-set point clouds by transforming them about the X-axis at four angles in radians (0.3, 0.6, 0.9, 2).
Performed inference using pretrained models (best_model.pt for both classification and segmentation) on each rotated version of the dataset.
Evaluated performance consistency by computing test accuracy at each angle and visualizing outputs on six fixed benchmark objects (indices: 562, 397, 308, 434, 490, 413).

b. Interpretation :

Classification performance degrades sharply under rotation, showing that the model relies on orientation-specific global features learned from upright training examples.
Accuracy collapses at higher rotations because the model fails to recognize objects when their canonical structure is altered, confirming that the global feature extraction pipeline is not rotation-invariant.
Segmentation remains significantly more stable, with only mild accuracy loss across rotations; the model continues to identify object parts reliably due to its focus on local geometric patterns rather than global orientation.

c. Classification Results :

Rotation (rad)	Success Rate	Object 1	Object 2	Object 3	Object 4
Ground Truth (0)	0.9832

Rotation (rad)	Success Rate	Object 1	Object 2	Object 3	Object 4
0.3	0.9296

Rotation (rad)	Success Rate	Object 1	Object 2	Object 3	Object 4
0.6	0.6804

Rotation (rad)	Success Rate	Object 1	Object 2	Object 3	Object 4
0.9	0.3462

Rotation (rad)	Success Rate	Object 1	Object 2	Object 3	Object 4
2.0	0.3168

d. Segmentation Results :

0.0 rad	Success	Object 1	Object 2	Object 3	Object 4
Ground Truth
Prediction	0.8986

0.3 rad	Success	Object 1	Object 2	Object 3	Object 4
Ground Truth
Prediction	0.8133

0.6 rad	Success	Object 1	Object 2	Object 3	Object 4
Ground Truth
Prediction	0.7133

0.9 rad	Success	Object 1	Object 2	Object 3	Object 4
Ground Truth
Prediction	0.6037

2.0 rad	Success	Object 1	Object 2	Object 3	Object 4
Ground Truth
Prediction	0.2654

EXPERIMNET 2 : NUMBER OF POINTS

a. Procedure :

Generated multiple point-density by randomly subsampling each object's 10,000-point cloud down to 500, 1000, 2000, and 5000 points.
Ran inference with the pretrained models on each density level.
Measured accuracy at varying sparsity levels and visualized predictions on the same six reference objects (indices: 562, 397, 308, 434, 490, 413).

b. Interpretation :

For classification, the model maintains strong accuracy even as input point density is reduced, showing only a minor decline, which indicates it effectively captures the overall geometric structure of objects. The network can extract global features from sparse point clouds, enabling reliable performance even with fewer sampled points.
The segmentation model remains effective with fewer input points, as global max-pooling preserves meaningful point-wise feature representations despite lower sampling. Overall, while classification shows greater robustness, segmentation handles sparse inputs better than it does rotated point clouds, according to both visual and quantitative results.

c. Classification Results :