Assignment 5: PointNet for Classification and Segmentation

Interpretation

The PointNet segmentation model achieves good overall performance (90.45% accuracy) by combining local point features with global context. Key observations:

• Good Cases (Objects 0, 1, 2, 3, 5): The model correctly segments most parts of the chairs, with clear boundaries between different semantic regions (seat, backrest, legs, etc.). The global feature provides useful context for disambiguating similar local geometries.
• Failure Case - Object 4: This chair exhibits a notable segmentation error where the model incorrectly predicts parts of the flat chair seat as chair arms (armrests). This failure can be attributed to two main factors:
  • Dataset Bias: The training data likely contains a majority of armchairs (as seen in Objects 0-3), causing the model to develop a bias toward predicting arm structures on the sides of chairs.
  • Poor Local Information Extraction: PointNet's reliance on global max pooling limits its ability to capture fine-grained local geometric differences between chair seats and armrests. The slightly distinguished local geometry of these regions is not adequately captured by the model's point-wise features, leading to confusion between semantically different but geometrically similar regions.
• Limitations: PointNet's lack of explicit local neighborhood modeling means it may miss fine-grained details at region boundaries. The max pooling operation aggregates information globally but may not preserve important local spatial relationships needed for precise segmentation, particularly when distinguishing between parts with subtle geometric differences.

Analysis

Classification Results:

• PointNet++ achieves a modest improvement (+0.42%) over PointNet for classification. The hierarchical local feature learning helps capture more discriminative features, especially for objects with complex geometric structures.
• The improvement is relatively small because PointNet already performs very well (98.22%), leaving little room for improvement. However, PointNet++'s local aggregation may help with edge cases.

Segmentation Results:

• Our implementation of PointNet++ performs slightly worse (-2.01%) than PointNet for segmentation. This is unexpected and can be attributed to several factors:
• Simplified Architecture: The implemented PointNet++ segmentation model differs significantly from the original paper (Qi et al., 2017). While the classification model follows the hierarchical Set Abstraction architecture from the paper, the segmentation model uses a simplified approach that does NOT implement the full feature propagation (FP) architecture.
• Missing Components: The original PointNet++ segmentation uses an hourglass architecture: SA↓ → SA↓ → SA↓ → FP↑ → FP↑ → FP↑. Our implementation instead uses: per-point MLP → local k-NN aggregation → global feature → decoder. This means we're missing:
  • Hierarchical downsampling with Set Abstraction layers
  • Feature Propagation (upsampling) layers with interpolation
  • Skip connections between encoder and decoder
  • Multi-scale grouping at different resolutions
• Why This Simplification? The full PointNet++ segmentation is complex to implement without custom CUDA kernels for ball query and efficient FPS. Our simplified version captures the "spirit" of locality through k-NN grouping but doesn't have the hierarchical structure.
• Training Issues: The more complex architecture with local neighborhoods requires more careful hyperparameter tuning. The model may not have been fully optimized for this specific task.

Key Takeaway: The classification PointNet++ follows the paper's architecture closely and shows improvement, while the segmentation PointNet++ is a simplified "local-enhanced PointNet" rather than true hierarchical PointNet++, explaining its lower performance.