16-825: Learning for 3D Vision

Assignment 5 - Fall 2026

Karthik Srinivasan (ksriniv2)

Compute: RTX 5090

Contents

Q1. Classification Model

Test accuracy

I used the default parameters. The best test accuracy was 97.8%, at epoch 164.

Visualizations

true chair, predicted chair
true chair, predicted chair
true chair, predicted chair
true chair, predicted chair
true chair, predicted chair
true chair, predicted chair
true chair, predicted chair
true chair, predicted chair
true vase, predicted vase
true vase, predicted vase

Failure predictions

No chair is misclassified (all 617 true chairs are correct), so the chair case below is a vase predicted as a chair.

true vase, predicted chair
true vase, predicted chair
true vase, predicted lamp
true vase, predicted lamp
true lamp, predicted vase
true lamp, predicted vase

The vase predicted as a chair is a wide, boxy base with a small plant on top. The block-shaped base with a flat bottom looks like a chair seat on a base, so the global shape matches chair features. The vase predicted as a lamp is a tall, thin vase with a flower stem rising from it. The stem and the narrow neck look like a lamp pole, so the model matches it to a lamp. The lamp predicted as a vase has a round body with a short shade on top. The large round body looks like a vase, and the small shade is easy to miss. In all three cases the object has shape features that belong to another class, so PointNet's global max-pooled feature lands closer to the wrong class.

Q2. Segmentation Model

Test accuracy

I used the default parameters. The best test accuracy was 90.1%, at epoch 109.

Segmentation results

3 randomly chosen objects and the 2 worst objects.

Object 308: accuracy 0.9272

gt
ground truth
pred
prediction

Object 608: accuracy 0.9874

gt
ground truth
pred
prediction

Object 98: accuracy 0.9646

gt
ground truth
pred
prediction

Object 351 (bad prediction): accuracy 0.4576

gt
ground truth
pred
prediction

Object 163 (bad prediction): accuracy 0.4754

gt
ground truth
pred
prediction

The model does well on regular chairs. The parts of these chairs have well-defined boundaries, where surfaces in different plane directions meet at a bend or corner, so the predicted parts match the ground truth almost exactly and the few errors sit right at those boundaries.

The two bad cases fail where the ground-truth boundaries are not marked by a bend in the geometry:

In both cases nothing in the geometry marks where one part ends, so the model falls back on the typical layout of a chair.

Q3. Robustness Analysis

Both experiments use the best models from Q1 and Q2.

Experiment 1: rotating the test point clouds

Procedure: I rotate every test point cloud about the z axis by a fixed angle and measure test accuracy on the full test set.

Rotation (deg)ClassificationSegmentation
0 (Q1 / Q2)0.97800.9014
300.82480.7116
600.49840.5102
900.29490.3560
1800.60860.3335

For the visualizations I use the same chair, vase and lamp for classification (all correct at 0°) and test object 0 for segmentation.

Classification, rotation 30°

rot30_idx322_gt-chair_pred-lamp.gif
30°: true chair, predicted lamp
rot30_idx688_gt-vase_pred-vase.gif
30°: true vase, predicted vase
rot30_idx719_gt-lamp_pred-lamp.gif
30°: true lamp, predicted lamp

Classification, rotation 90°

rot90_idx322_gt-chair_pred-lamp.gif
90°: true chair, predicted lamp
rot90_idx688_gt-vase_pred-vase.gif
90°: true vase, predicted vase
rot90_idx719_gt-lamp_pred-vase.gif
90°: true lamp, predicted vase

Classification, rotation 180°

rot180_idx322_gt-chair_pred-chair.gif
180°: true chair, predicted chair
rot180_idx688_gt-vase_pred-lamp.gif
180°: true vase, predicted lamp
rot180_idx719_gt-lamp_pred-lamp.gif
180°: true lamp, predicted lamp

Segmentation, rotation 30°

ground truth
prediction

Segmentation, rotation 90°

ground truth
prediction

Segmentation, rotation 180°

ground truth
prediction

Experiment 2: number of points per object

Procedure: I give the model a random subset of points per test object instead of all 10000 and measure test accuracy on the full test set.

PointsClassificationSegmentation
10000 (Q1 / Q2)0.97800.9014
50000.97480.9006
10000.97270.8881
5000.96010.8668
1000.92760.7860

Classification, 1000 points

pts1000_idx322_gt-chair_pred-chair.gif
1000 pts: true chair, predicted chair
pts1000_idx688_gt-vase_pred-vase.gif
1000 pts: true vase, predicted vase
pts1000_idx719_gt-lamp_pred-lamp.gif
1000 pts: true lamp, predicted lamp

Classification, 100 points

pts100_idx322_gt-chair_pred-chair.gif
100 pts: true chair, predicted chair
pts100_idx688_gt-vase_pred-vase.gif
100 pts: true vase, predicted vase
pts100_idx719_gt-lamp_pred-lamp.gif
100 pts: true lamp, predicted lamp

Segmentation, 1000 points

ground truth
prediction

Segmentation, 100 points

ground truth
prediction

Q4. Bonus Question - Locality

Model

I implemented DGCNN. Each EdgeConv layer picks the 10 nearest neighbors of every point in feature space, combines each point with its neighbors, runs a shared linear layer with BatchNorm and LeakyReLU, and max-pools over the neighbors. So every point gets some local context, which PointNet does not have.

I trained with batch size 16, using 5000 points per object for classification and 10000 for segmentation. Classification has wider EdgeConv layers (up to 256 channels, against 64 for segmentation) and ran out of GPU memory at 10000 points, so I used 5000. Everything else is the same as Q1 and Q2.

Test accuracy

TaskPointNet (Q1 / Q2)DGCNN
Classification0.9748 (5000 pts)0.9811 (5000 pts, epoch 97)
Segmentation0.9014 (10000 pts)0.9197 (10000 pts, epoch 136)

I visualize the same objects as in Q1 and Q2 and compare the predictions. I check whether DGCNN fixes the objects that failed earlier, and whether it fails on anything new that PointNet got right.

Classification - Visualization

Same objects as Q1 (all correct, as in Q1)

true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true chair, DGCNN predicts chair
true vase, DGCNN predicts vase
true vase, DGCNN predicts vase

Q1 failures that DGCNN gets right

true vase, DGCNN predicts vase
true vase, DGCNN predicts vase
true lamp, DGCNN predicts lamp
true lamp, DGCNN predicts lamp
true vase, DGCNN predicts vase
true vase, DGCNN predicts vase

New DGCNN failures (PointNet was correct)

true chair, DGCNN predicts lamp
true chair, DGCNN predicts lamp
true lamp, DGCNN predicts vase
true lamp, DGCNN predicts vase

Wrong in both models

true vase, DGCNN predicts lamp
true vase, DGCNN predicts lamp
true lamp, DGCNN predicts vase
true lamp, DGCNN predicts vase

Segmentation - Visualization

Object 308: PointNet 0.9272, DGCNN 0.9685

ground truth
ground truth
PointNet
PointNet
DGCNN
DGCNN

Object 608: PointNet 0.9874, DGCNN 0.9852

ground truth
ground truth
PointNet
PointNet
DGCNN
DGCNN

Object 98: PointNet 0.9646, DGCNN 0.9562

ground truth
ground truth
PointNet
PointNet
DGCNN
DGCNN

Object 351: PointNet 0.4576, DGCNN 0.4821

ground truth
ground truth
PointNet
PointNet
DGCNN
DGCNN

Object 163: PointNet 0.4754, DGCNN 0.4906

ground truth
ground truth
PointNet
PointNet
DGCNN
DGCNN