Q1. Classification Model

Visualize examples of correct and incorrect classifications for each class.
Right: Correct class | Left: Incorrect class

---

Chair

Correct

Incorrect

Lamp

Correct

Incorrect

Vase

Correct

Incorrect

---

Observations:

• The model typically fails on more complicated or visually similar examples.
• Chair: Fails when the chair resembles a vase or has a unique shape.
• Lamp: Fails when the lamp is a chandelier, likely due to its rarity in the training data.
• Vase: Fails when it is boxy and closely resembles a chair.

Test Accuracy: 0.877

Q2. Segmentation Model

Visualize segmentation results of at least 5 objects (including 2 bad predictions) with corresponding ground truth, report the prediction accuracy for each object, and provide a brief interpretation.

Note: Left: Ground truth | Right: Model prediction

---

Example 1 — Accuracy: 0.8443 (ID: 0)

---

Example 2 — Accuracy: 0.8433 (ID: 1)

---

Example 3 — Accuracy: 0.6094 (ID: 2)

---

Example 4 — Accuracy: 0.4169 (ID: 4)

---

Example 5 — Accuracy: 0.8953 (ID: 8)

---

Final Test Accuracy: 75.2%

The model performs well on most parts of the chair but struggles on thin structures (e.g., armrests), capturing general shapes but missing finer details.

Q3. Robustness Analysis

Conduct 2 experiments to analyze the robustness of your learned model. Each experiment is worth 10 points, for a maximum of 20 points.

---

Experiment 1: Rotation

Classification Task

Chair

Lamp

Vase

Test accuracy: 0.574

After applying a 65° roll, pitch, and yaw rotation in the local frame, the classifier performance drops from 87.7% → 57.4%, indicating that the model is not robust to rotations. Although the model learns the most important points for classification, when these points are rotated, their positions shift relative to the model's learned features, causing the model to fail.

---

Segmentation Task

Test accuracy: 0.315
Object accuracy: 0.198

With 65° rotation, segmentation accuracy drops from 75.2% → 31.2%, showing low robustness.

Test accuracy: 0.683
Object accuracy: 0.749

For 20° rotation, accuracy drops less, showing some tolerance to smaller rotations.

Similarly to the classification task, the segmentation model learns the most important global points to classify each individual point. When the point cloud is rotated relative to the model's learned features, these points shift, causing the model to fail.

---

Experiment 2: Number of Points

Classification Task

Chair

Lamp

Vase

With 100 sampled points, classifier performance remains stable (86.1% vs. 87.7% original).
This is because the model’s max pooling layer captures the most prominent features, making predictions robust even with fewer points.

---

Segmentation Task

Example 1

Ground Truth

Prediction

Object accuracy: 0.8

Example 2

Ground Truth

Prediction

Object accuracy: 0.78

With 100 points, total segmentation accuracy stays at 75.1% vs. 75.2% original.
The model is robust because it learns the most informative points, maintaining context even with fewer samples.

---