Parth Burujwale Andrew ID:
pburujwa
16-825: Learning for 3D Vision
The saved best baseline classifier achieved 98.1112% test accuracy: 935 correct predictions out of 953 objects. There were 18 errors: two chairs, eight vases, and eight lamps.
The following six test objects were selected with a fixed random seed. Each figure includes the ground-truth and predicted categories.
| Test object 90 | Test object 156 | Test object 183 |
|---|---|---|
![]() |
![]() |
![]() |
| Test object 359 | Test object 486 | Test object 900 |
|---|---|---|
![]() |
![]() |
![]() |
| Ground truth | Prediction | Test object |
|---|---|---|
| Chair | Lamp | 406 |
| Vase | Lamp | 650 |
| Lamp | Vase | 750 |
| Chair misclassified as lamp | Vase misclassified as lamp | Lamp misclassified as vase |
|---|---|---|
![]() |
![]() |
![]() |
Chair 406: The displayed object has a broad, low surface and thin supports, without a prominent conventional backrest in this view. Its unusual shape may make the category harder to distinguish from other supported structures.
Vase 650: Its irregular, multipart silhouette differs from a simple upright vessel. Some rounded and elongated components could resemble features found in decorative lamps.
Lamp 750: The displayed object has a container-like cylindrical body, making vase versus lamp confusion plausible when distinctive lamp components are less apparent.
These are visual interpretations of the failures, rather than verified explanations of the model’s internal decisions. The baseline compresses the object into a global maximum-pooled descriptor, which may lose some relationships between nearby parts.
The best baseline segmentation checkpoint achieved approximately 89.50% overall point accuracy.
Objects 426 and 10 were selected as poor baseline examples. Objects 102, 120, and 233 were selected randomly from the remaining test objects using seed 825. Ground truth and predictions use the same part-color convention within these GIFs.
| Test object | Ground truth | Baseline prediction |
|---|---|---|
| 426 — poor example | ![]() |
![]() |
| 10 — poor example | ![]() |
![]() |
| 102 — random example | ![]() |
![]() |
| 120 — random example | ![]() |
![]() |
| 233 — random example | ![]() |
![]() |
The common comparison evaluator measured the following baseline accuracies on these objects:
| Test object | Baseline point accuracy |
|---|---|
| 426 | 24.97% |
| 10 | 45.51% |
| 102 | 98.15% |
| 120 | 95.59% |
| 233 | 95.10% |
The three random examples have point accuracies above 95%, while the two selected poor examples expose substantial failures. This contrast shows that aggregate accuracy does not fully describe performance on individual shapes.
The baseline combines each point’s features with a global object descriptor, but it does not explicitly encode local neighborhoods. Local part boundaries and relationships between nearby surfaces are therefore learned indirectly. This motivates the neighborhood-based model in Q4. Overall point accuracy also gives more weight to parts containing more points; it is not a per-part IoU metric.
Two experiments tested both trained baseline models without retraining: rotations around the Y axis and reductions in input point count. Evaluation used the saved best checkpoints in evaluation mode. Classification reports object accuracy; segmentation reports point accuracy.
Every test point cloud was rotated by 0°, 15°, 30°, 60°, or 90° around the Y axis, retaining all 10,000 points. Class labels and the segmentation labels attached to individual points were unchanged.
For angle θ, the rotation was:
\[ R_y(\theta)= \begin{bmatrix} \cos\theta & 0 & \sin\theta \\ 0 & 1 & 0 \\ -\sin\theta & 0 & \cos\theta \end{bmatrix}. \]
| Y-axis rotation | Classification accuracy | Classification drop | Segmentation accuracy | Segmentation drop |
|---|---|---|---|---|
| 0° | 98.1112% | 0.00 pp | 89.4990% | 0.00 pp |
| 15° | 96.4323% | 1.68 pp | 85.1741% | 4.32 pp |
| 30° | 83.5257% | 14.59 pp | 76.4612% | 13.04 pp |
| 60° | 45.0157% | 53.10 pp | 64.3310% | 25.17 pp |
| 90° | 50.4722% | 47.64 pp | 57.1058% | 32.39 pp |
Here and below, pp denotes percentage points relative to the unmodified baseline.
| Original orientation | 90° rotation |
|---|---|
![]() |
![]() |
| Original orientation | 90° rotation |
|---|---|
![]() |
![]() |
Both models are sensitive to orientation. Classification drops sharply beyond 15°, reaching 45.02% at 60°. Its recovery to 50.47% at 90° shows that degradation is not strictly monotonic with angle. Segmentation decreases across the tested angles, reaching 57.11% at 90°.
Permutation invariance does not imply rotation invariance: rotating an object changes its XYZ coordinates and the learned responses to those coordinates. The results suggest reliance on orientation patterns present in the training data. Rotation augmentation or explicit rotation-aware design would be possible follow-up experiments; they were not evaluated here.
Each model was evaluated using 10,000, 5,000, 1,000, 500, or 100 points per object. A fixed random permutation with seed 825 defined nested subsets for the reduced point counts within each task. The same point indices were used across objects in that task, and segmentation targets were subsampled with their corresponding points.
The 10,000-point setting used the original complete input. No retraining or additional coordinate normalization was applied.
| Points per object | Classification accuracy | Classification drop | Segmentation accuracy | Segmentation drop |
|---|---|---|---|---|
| 10,000 | 98.1112% | 0.00 pp | 89.4990% | 0.00 pp |
| 5,000 | 98.1112% | 0.00 pp | 89.5336% | −0.03 pp |
| 1,000 | 97.6915% | 0.42 pp | 89.0702% | 0.43 pp |
| 500 | 97.4816% | 0.63 pp | 87.5361% | 1.96 pp |
| 100 | 93.8090% | 4.30 pp | 70.8088% | 18.69 pp |
| 10,000 input points | 100 input points |
|---|---|
![]() |
![]() |
| 10,000 input points | 100 input points |
|---|---|
![]() |
![]() |
Classification is relatively robust to reduced point counts: it retains 97.48% accuracy with 500 points and 93.81% with 100 points. Coarse object shape remains informative even when many points are removed.
Segmentation is more affected at extreme sparsity. Its accuracy falls by 18.69 percentage points at 100 points, consistent with needing sufficient geometric coverage to distinguish parts. At reduced point counts, segmentation accuracy is measured on the retained points, rather than on all original points.
The slight increase at 5,000 points is only about 0.03 percentage points. Since each reduced-count setting uses one deterministic subset, this does not establish that subsampling improves segmentation generally.
The bonus model is PointNet++ inspired, using farthest-point sampling and k-nearest-neighbor grouping rather than reproducing the original paper’s radius-based grouping architecture exactly.
| Stage | Operation | Output feature width |
|---|---|---|
| Input | 10,000 XYZ points | 3 |
| Local stage 1 | Sample 512 centers; group 32 neighbors; shared MLP 6 → 64 → 64 → 128; max-pool neighbors | 128 per center |
| Local stage 2 | Sample 128 centers; group 32 neighbors; shared MLP 131 → 128 → 128 → 256; max-pool neighbors | 256 per center |
| Global stage | Concatenate center coordinates; shared MLP 259 → 256 → 512; max-pool centers | 512 per object |
Each local group includes coordinates relative to its center and the neighbors’ input features. The first stage uses absolute XYZ coordinates as its initial features, alongside the relative coordinates. The second stage groups the learned first-stage features.
Classification: The 512-dimensional global feature passes through a 512 → 256 → 128 → 3 classifier. Hidden layers use batch normalization, ReLU, and dropout with probability 0.3.
Segmentation: The global descriptor is concatenated with coarse local features. Three-neighbor interpolation propagates features from 128 centers to 512 centers, then to all input points. Interpolation weights are proportional to inverse squared Euclidean distance, with a small numerical floor. Skip connections fuse finer features at each stage. A pointwise head predicts six part logits per original point.
Both locality models were trained from scratch for 250 epochs with cross-entropy loss and Adam at learning rate 0.001. Classification used batch size 32; segmentation used batch size 4. All 10,000 input points were provided to the models before internal sampling.
Classification training took 49 minutes 25 seconds
and segmentation training took 2 hours 3 minutes 29
seconds on an AWS g5.xlarge instance with an
NVIDIA A10G. The environment used Python 3.10, PyTorch 2.7.1 with CUDA
12.8, and PyTorch3D 0.7.9.
| Task | Baseline accuracy | Locality accuracy | Gain |
|---|---|---|---|
| Classification | 98.1112% | 98.3211% | +0.21 pp |
| Segmentation | 89.4991% | 91.7682% | +2.27 pp |
The classification model correctly labels 937 of 953 objects, two more than the baseline. The segmentation gain is larger in percentage-point terms.
These are the same three baseline failures shown in Q1. Each comparison labels the ground truth and the prediction from each model.
The same five Q2 objects are shown with ground truth, baseline prediction, and locality prediction. Within each comparison, all panels use identical displayed points and viewpoints. Gray, magenta, cyan, yellow, blue, and red represent part IDs 0–5, respectively.
| Test object | Baseline accuracy | Locality accuracy | Change |
|---|---|---|---|
| 426 | 24.97% | 61.78% | +36.81 pp |
| 10 | 45.51% | 88.45% | +42.94 pp |
| 102 | 98.15% | 98.03% | −0.12 pp |
| 120 | 95.59% | 95.67% | +0.08 pp |
| 233 | 95.10% | 91.48% | −3.62 pp |
The locality model improves overall performance on both tasks in these runs. Explicit neighborhood aggregation provides information about nearby surfaces and their relative arrangement before global pooling or feature propagation.
The two difficult segmentation examples improve substantially: object 426 gains 36.81 percentage points and object 10 gains 42.94 points. However, locality does not improve every example. Object 233 loses 3.62 points, and object 426 remains challenging at 61.78% accuracy.
The observed gains cannot be attributed exclusively to locality: the architecture also changes feature widths, parameterization, and the segmentation decoder. These are single-run comparisons without repeated-seed uncertainty estimates. Following the provided training loop, best checkpoints were selected using the supplied test split, so these numbers are not an independent evaluation on a separate untouched holdout set.