The following examples illustrate correct classifications by the model:
Below are some misclassified examples, along with their true and predicted labels:
The model demonstrates high accuracy on the test set and effectively distinguishes between objects of different shapes, as shown in the successful predictions.
In the first failure case, the chair appears out of distribution, resembling a blender-chair, which likely caused confusion. The second and last failure cases involve highly symmetric objects, making it difficult to differentiate between lamps and vases—even for humans.
The following examples show correct segmentations by the model:
Below are examples with low segmentation accuracy:
The segmentation model demonstrates strong performance on the test set, accurately identifying chair parts across a range of shapes. Accuracy is highest for simpler chairs that include only a few of the six possible part classes—typically the back, seat, and legs. In these cases, the model achieves near-perfect segmentation, indicating that lower part complexity facilitates more reliable predictions.
In the failure cases, several challenges emerge. The first example involves a chair where the "back", "legs," and "arms" form a single continuous structure; while the model correctly identifies the legs, the ground truth assigns distinct labels to the arms and back, making boundary delineation difficult. The second example features an asymmetric chair with only one arm, which is inherently challenging to segment based solely on point positions. The third example likely suffers from incorrect ground truth labels, leading to a low accuracy score despite a plausible prediction by the model.
To assess the robustness of the trained models, I conducted two experiments for both classification and segmentation:
Evaluating the classification model with varying numbers of input points reveals that accuracy declines as the number of points decreases, which is expected. Additionally, the random sampling of points has a significant impact on accuracy, particularly when fewer points are used. As illustrated in the plot, changing the random seed can lead to noticeable fluctuations in accuracy, even when the number of points remains constant.
Some examples of the prediction using different number of points
Left: successful examples. Right: failed examples.
Left: successful examples. Right: failed examples.
Left: successful examples. Right: failed examples.
Evaluating the classification model on rotated input point clouds shows that the model lacks robustness to rotation, which is expected since the objects have a canonical upright orientation (e.g., chairs are not expected to appear upside down). Interestingly, classification accuracy increases slightly at 180 degrees, likely due to the vertical symmetry of certain objects such as vases. The plot also reveals a form of horizontal symmetry: rotating by 45 degrees yields similar results to rotating by 315 degrees.
Some examples of the prediction using different number of points
When evaluating the segmentation model with different numbers of input points, accuracy decreases as the number of points is reduced, which is expected. However, the model still achieves high accuracy (over 90%) with as few as 3000 points. This is reasonable since each point is classified independently among chair parts, and the main cause of accuracy reduction with fewer points is uneven subsampling rather than insufficient information. Consequently, random sampling of points has minimal impact on segmentation accuracy, as shown in the plot.
Some examples of the prediction using different number of points
Below are examples with low segmentation accuracy:
Below are examples with low segmentation accuracy:
Below are examples with low segmentation accuracy:
Evaluating the segmentation model on rotated input point clouds demonstrates that the model is not robust to rotation. This is expected, as the objects are presented in a canonical upright orientation (e.g., chairs are not expected to appear upside down), and each point is classified solely based on its 3D position. As a result, points located at the bottom are consistently labeled as legs, regardless of the object's orientation. Similar to the classification model, the accuracy-versus-rotation plot reveals a form of horizontal symmetry: rotating by 45 degrees produces results comparable to rotating by 315 degrees.
Here are some examples of model predictions on point clouds rotated by different angles:
