16-825: Learning for 3D Vision · Kashu Yamazaki
Q1 Classification · Q2 Segmentation · Q3 Robustness · Q4 DGCNN
The classifier is PointNet without the T-Net blocks. A shared per-point MLP with widths 3–64–64–64–128–1024 (each layer a 1×1 convolution, BatchNorm and ReLU) is followed by a max pool over the points and a 1024–512–256–3 fully connected head, with BatchNorm and ReLU after the first two layers and dropout 0.3 before the last. I trained it for 100 epochs with Adam (lr 1e-3, batch 32, no schedule) on all 10000 points of each object. The run took 22.7 minutes on an RTX 3080, and the best epoch was 54.
Checkpoint selection. The provided train.py keeps the checkpoint with the highest
test accuracy, so the accuracies in Q1, Q2 and Q4 come from an epoch selected on the test set and are
slightly optimistic compared with selection on a held-out split. Classification test accuracy first
passed 0.95 at epoch 6 (epochs count from 0), and 65 of the 80 epochs from epoch 20 on scored 0.96 or higher.
Segmentation test accuracy varied between 0.804 and 0.902 from epoch 100 on.
Test accuracy is 0.9759 (930 of 953 objects), with all 10000 points per object. Vases have the highest error rate and are also the smallest class in training (741 of 6784 objects).
| Class | Test objects | Correct | Accuracy | Misclassified as |
|---|---|---|---|---|
| Chair | 617 | 616 | 0.998 | 1 lamp |
| Vase | 102 | 87 | 0.853 | 13 lamp, 2 chair |
| Lamp | 234 | 227 | 0.970 | 5 vase, 2 chair |
Two objects per class, drawn with np.random.default_rng(0). Each GIF is a 30-frame
turntable, and each point is coloured by its normalised xyz position so that depth and separate parts
are visible. The number in brackets is the predicted class probability.






Object 543 is the only misclassified chair. The vase and lamp failures shown are two of the 15 misclassified vases and two of the 7 misclassified lamps.





Chair 543 is a ladder-back chair whose backrest is several times taller than the seat is wide, so most of its points lie on a tall, thin vertical strip above a small base. That outline is closer to a floor lamp than to a typical chair, and the model gave lamp 0.65.
Vase 651 is an orchid in a shallow bowl. The flower stalk curves up and over the bowl and ends in a wide flower head, which has the layout of a desk lamp: round base, bent arm, shade at the end. Vase 644 in the random set, a leafy plant in a pot, is the same kind of error. Vases that hold plants have spiky geometry at the top that resembles lamp arms or shades.
Vase 671 is a wide, low planter tray with small plants along the top. Most vases in the data are tall and closed, while this one is a flat rectangular slab about as high as a seat, and the model labelled it chair.
Lamp 827 is a drum shade on a short base with no visible stem. Its points form a closed shell that is close to rotationally symmetric, which is the usual vase shape. Lamp 922 is a candelabra with several curved arms that end in small shades on a small base, and it looks like stems in a narrow vase.
Of the 23 test errors, 18 are confusions between lamps and vases (13 vases called lamps, 5 lamps called vases). Both classes contain rotationally symmetric bodies and thin upright parts. PointNet reduces the object to one max-pooled feature, which encodes which local shapes are present but not how they connect, so a stem in a vase and a lamp arm give similar features.
The segmentation network is PointNet without T-Nets. A per-point MLP 3–64–64 gives a 64-d local feature, and a second MLP 64–64–128–1024 followed by a max pool gives a 1024-d global feature. The global feature is copied to every point and concatenated with its local feature (1088-d), and a per-point MLP 1088–512–256–128 and a 128–6 convolution give the per-point logits. I trained it for 150 epochs with Adam (lr 1e-3, no schedule) at batch 16, on all 10000 points of each object; batch 32 peaked at 8.96 GB, too close to the 10 GB of the RTX 3080. The run took 64.1 minutes, and the best epoch was 136.
Test accuracy is 0.9021, counted over all 6,170,000 test points (617 chairs of 10000 points). Per-object accuracy ranges from 0.470 (object 235) to 0.997 (object 471).
In the GIFs, cyan is the back, red the seat, blue the legs or base, and yellow the arms. The provided
viz_seg colours label 0 white, the same as the background, so label 0 points do not show.
Label 0 is 0.33% of test points, and none of the five objects below has a label 0 or label 1 point in
either the ground truth or the Q2 prediction. Label 0 predictions appear later on the page, on
chair 392 under yaw in Q3 and on chair 426 in Q4, and those captions give the counts.
| Object | Chosen as | Per-object accuracy |
|---|---|---|
| 392 | random | 0.8060 |
| 315 | random | 0.9426 |
| 523 | random | 0.8629 |
| 235 | worst in the test set | 0.4695 |
| 351 | second worst | 0.4719 |
The random objects were drawn with np.random.default_rng(0).choice(617, 3, replace=False).










Object 315 is an office chair with a five-star base. The prediction matches the ground truth except along the seat–back join and where the arms meet the seat. Object 392 is a rocking chair; the model extends the seat label up into the lower part of the backrest and onto the arm supports, and gets the rockers and the back right. Object 523 is a chair without arms whose rear legs continue past the seat, and its errors are on the leg tops next to the seat and on the seat edge. In all three, the errors sit at part boundaries. A point's local feature in this network comes from its own coordinates only, so points just either side of a boundary get nearly the same feature.
Object 235 is a sofa-like armchair with closed sides and a skirt down to the floor. The ground truth labels the side panels and much of the box as arms (6165 of 10000 points) and the lower front as seat. The model labels the bottom of the box as legs and the cushion area as seat, which is where those parts are on a typical chair. Object 351 is a cylindrical barrel chair with a cut-out in the front. Its ground truth labels the shell as arms and back down to the floor and has no leg points, while the model splits the shell by height, with legs on the bottom half and back on the top half. Neither chair has visible legs, and on both the model assigns labels by height and position as it would on an ordinary four-legged chair.
robustness.py evaluates the Q1 and Q2 checkpoints under three perturbations of the test
set, without retraining. Every setting starts from the same subsample as eval_cls.py and
eval_seg.py (np.random.seed(0), then np.random.choice(10000, N),
applied to every object and to the segmentation labels), so the unperturbed rows reproduce Q1 (0.9759)
and Q2 (0.9021). Classification accuracy is over the 953 test objects and segmentation accuracy over
all points of the 617 test chairs.
The data are y-up: chair legs sit at mean y = −0.47 and backs at y = +0.34, and chairs face +z. The grids below render each perturbed cloud from one fixed camera (elevation 20°, azimuth 30°). Classification grids show the first test object of each class (0 chair, 617 vase, 719 lamp), coloured by the unperturbed xyz so that colours follow the same points across columns; a red title is a wrong prediction. Segmentation grids show chair 392 from Q2, ground truth above the prediction, with that object's accuracy in the title.
All points are rotated about the origin by 0 to 180 degrees about one of two axes, with all 10000 points kept. Rotation about x tips the object forward or backward, so the up axis no longer points up. Rotation about y (yaw) spins the object in place on the floor, which is the orientation change that a real scan of an upright object would show.
| Angle (deg) | Cls, x-axis | Cls, y-axis | Seg, x-axis | Seg, y-axis |
|---|---|---|---|---|
| 0 (Q1/Q2) | 0.9759 | 0.9759 | 0.9021 | 0.9021 |
| 15 | 0.9234 | 0.9706 | 0.8235 | 0.8681 |
| 30 | 0.8604 | 0.9244 | 0.7060 | 0.8087 |
| 45 | 0.8101 | 0.8416 | 0.6118 | 0.7289 |
| 60 | 0.6695 | 0.7450 | 0.4662 | 0.6463 |
| 90 | 0.3242 | 0.6925 | 0.2211 | 0.5979 |
| 135 | 0.6600 | 0.3221 | 0.2153 | 0.5136 |
| 180 | 0.6590 | 0.4176 | 0.2522 | 0.6518 |
Per-class classification accuracy at selected angles, from the confusion matrix over the full test set (617 chairs, 102 vases, 234 lamps). Answering “chair” for every object would score 0.647.
| Rotation | Chair | Vase | Lamp | Misclassified chairs |
|---|---|---|---|---|
| x, 90° | 0.165 | 0.373 | 0.722 | 381 vase, 134 lamp |
| x, 180° | 0.622 | 0.255 | 0.932 | 142 vase, 91 lamp |
| y, 90° | 0.577 | 0.824 | 0.940 | 201 vase, 60 lamp |
| y, 135° | 0.003 | 0.863 | 0.927 | 364 vase, 251 lamp |
| y, 180° | 0.151 | 0.824 | 0.944 | 25 vase, 499 lamp |




Turntable GIFs of chair 392 under rotation, next to the unrotated Q2 result. The GIF camera orbits the rotated cloud, so a tilted chair stays tilted throughout.








Neither model is rotation invariant. Without a T-Net, PointNet takes raw xyz coordinates, and the training objects are all in the same canonical orientation, so the features encode where parts are in the world frame. Tilting about x lowers accuracy fastest on both tasks. At 90° of tilt, classification falls to 0.32, below the 0.647 of a constant “chair” answer, and most chairs are called vases.
Under yaw, vases and lamps keep 0.82 to 0.94 accuracy at 90, 135 and 180°, as expected for shapes that are close to rotationally symmetric about y. Chairs drop to 2 of 617 correct at 135°, so the classifier recognises chairs partly by which way they face.
Segmentation degrades sooner than classification: at 15° of tilt it loses 0.08 against 0.05 for classification. The model labels points mostly by height. In the x-rotation grid, at 90 and 180° the lowest points are predicted as legs (blue) and the highest as back or arms whatever part they belong to, and at 180° the upside-down back is labelled as legs. Under yaw, segmentation accuracy recovers to 0.65 at 180° because the height order of the parts is unchanged and only front and back swap. At 90° of yaw the back is labelled as arms (yellow) and the arms as back (cyan), so the model also uses the z coordinate to find the back.
Each object is subsampled to N = 10000, 5000, 1000, 500, 100, 50 or 10 points with the same random indices for every object, with no other change. The models were trained on 10000 points.
| Points | Cls | Seg |
|---|---|---|
| 10000 (Q1/Q2) | 0.9759 | 0.9021 |
| 5000 | 0.9748 | 0.9023 |
| 1000 | 0.9727 | 0.8964 |
| 500 | 0.9654 | 0.8813 |
| 100 | 0.9192 | 0.7907 |
| 50 | 0.8048 | 0.7174 |
| 10 | 0.2707 | 0.5801 |


Accuracy is almost flat from 10000 down to 500 points: classification goes from 0.9759 to 0.9654 and segmentation from 0.9021 to 0.8813. The global feature is a max over points, so each channel depends on a few extreme points, and a random 5% subsample still lands near most of them. The loss becomes visible at 100 points and large at 10. At N = 10 the classifier predicts lamp for 929 of the 953 objects, including 593 of 617 chairs and all 102 vases; at N = 50 chairs are still 0.843 correct but vases drop to 0.176 (77 of 102 called lamp).
Segmentation loses more than classification at 100 and 50 points, but at N = 10 it still scores 0.58, because part labels correlate with height (legs low, back high) and a single point carries its own height. A max over fewer points is never larger in any channel, so at small N the pooled features fall below the values that the BatchNorm layers after the pool saw in training at N = 10000, which may account for part of the drop. I did not retrain at small N to check.
Independent Gaussian noise with standard deviation 0, 0.01, 0.02, 0.05 or 0.1 is added to each coordinate of all 10000 points (torch generator seeded with 0). Objects are scaled so that the farthest point is at distance 1 from the origin; over the classification test set the median extents are 0.94 (x), 1.59 (y) and 1.01 (z).
| Noise std | Cls | Seg |
|---|---|---|
| 0 (Q1/Q2) | 0.9759 | 0.9021 |
| 0.01 | 0.9727 | 0.9002 |
| 0.02 | 0.9664 | 0.8922 |
| 0.05 | 0.9381 | 0.8438 |
| 0.1 | 0.8877 | 0.7436 |


The noise in these two grids is drawn separately from the noise in the accuracy run, so a prediction in a grid can differ from that object's prediction in the run that produced the table.
Both models lose at most 0.01 up to std 0.02. At std 0.1, about 6% of a typical object's height and 10% of its width or depth, the shapes in the classification grid are visibly blurred and classification still holds 0.89, while segmentation falls to 0.74. Classification depends on the overall shape, which survives the blur, whereas a point's part label flips when the noise moves it across a part boundary.
I implemented DGCNN (Wang et al., “Dynamic Graph CNN for Learning on Point Clouds”). Its
building block is EdgeConv: for every point it finds the k = 20 nearest neighbours in the current
feature space and builds an edge feature [xj − xi, xi]
for each neighbour j. A shared 1×1 Conv2d, BatchNorm and LeakyReLU(0.2) MLP runs over those edges,
and a max over the 20 neighbours gives the new point feature. Each layer rebuilds the graph from its
input features, so later layers group points that are similar in feature space as well as close in 3D.
The neighbour search is torch.cdist and topk in PyTorch, with no compiled extensions.
dgcnn_cls (1.80 M parameters): four EdgeConv layers with 64, 64, 128 and 256
channels. Their outputs are concatenated (512 channels) and mapped to 1024 channels per point, then
max-pooled and average-pooled into a 2048-d global feature, followed by a
512–256–3 head with dropout 0.5.dgcnn_seg (0.64 M parameters): three EdgeConv layers ([64, 64], [64, 64] and [64];
the first two have two-layer edge MLPs), concatenated into 192 channels per point. A 1024-channel
max-pooled global feature is appended to every point (1216 channels), and the head is
256–256–128–6 with dropout 0.5. I left out the paper's spatial transform network and
its one-hot category vector, since this dataset contains only chairs.
The neighbour search builds an N×N distance matrix per object in every EdgeConv layer. At batch 32
and 10000 points that matrix is 32 × 10000² × 4 bytes = 12.8 GB in float32, more
than the 10 GB of the RTX 3080; at 1024 points it is 134 MB and at 2048 points 537 MB.
In training, each batch draws a fresh random subset of 1024 points per object for
classification and 2048 for segmentation, shared by all objects in the batch. The per-epoch test
accuracy that train.py uses to pick the checkpoint also comes from random subsets of the
same size. Training used the train.py defaults (Adam, lr 1e-3, batch 32, 250 epochs, no
schedule, no augmentation beyond the resampling) on the babel cluster, with classification and
segmentation on two NVIDIA L40S GPUs at the same time, about 19 s per epoch each. The best epochs
were 205 for classification and 198 for segmentation.
eval_local.py evaluates PointNet and DGCNN on the same points: 5 random subsets of point
indices (seeds 0 to 4), each shared by every object and both models, reported as mean ± std over
the 5 draws. Per-object numbers and GIFs use draw 0. The Q1/Q2 PointNet models were trained on 10000
points and are evaluated here on the same 1024 or 2048 points as DGCNN.
| Task | Points | PointNet (Q1/Q2) | DGCNN |
|---|---|---|---|
| Classification | 1024 | 0.9717 ± 0.0023 | 0.9832 ± 0.0007 |
| Classification | 10000 | 0.9759 | not run |
| Segmentation | 2048 | 0.9007 ± 0.0005 | 0.9205 ± 0.0003 |
| Segmentation | 10000 | 0.9021 | not run |
On classification draw 0, PointNet misclassifies 29 of 953 test objects: 2 chairs called lamps, 17 vases (15 called lamps, 2 called chairs) and 10 lamps (8 called vases, 2 called chairs). DGCNN misclassifies 17: 1 chair called a lamp, 6 vases called lamps and 10 lamps called vases.
Each GIF has three panels, left to right: ground truth, PointNet and DGCNN, all on the same 1024 points. Points are coloured by the class assigned to the whole object: blue chair, red vase, magenta lamp. The first three objects are Q1 failures.




Ground truth, PointNet and DGCNN, left to right, on the same 2048 points, with the Q2 colours. The numbers are per-object accuracy for PointNet and DGCNN. Label 0 is white here too; of these four chairs, only 426 has label 0 predictions (165 from PointNet, 15 from DGCNN).




Two of PointNet's 15 worst chairs, 426 and 212, go above 0.96 with DGCNN. Others barely move or drop slightly, for example chair 26 (0.534 to 0.531) and chair 331 (0.636 to 0.551).
DGCNN is more accurate on both tasks at the same number of points: 0.9717 to 0.9832 on classification and 0.9007 to 0.9205 on segmentation, with a smaller spread across the five point subsets. Moving PointNet from 10000 points to these subsets lowers it from 0.9759 to 0.9717 on classification and from 0.9021 to 0.9007 on segmentation, so most of the gap comes from the model and little from PointNet seeing fewer points than it was trained on.
In classification, nearly all of the gain is on vases: PointNet calls 15 vases lamps and DGCNN calls 6. Lamp errors stay at 10 for both models. PointNet sees each point only through its absolute coordinates before one global max pool. In DGCNN, each EdgeConv layer sees a point together with the offsets to its 20 neighbours, which describe the local surface shape (a thin stem, a curved wall, a flat seat). My reading is that this lets it keep most vases out of the lamp class. Lamps with large rounded shades (827, 750) still look like vases to it: all 10 of DGCNN's lamp errors are vase predictions, while most of PointNet's errors go from vase to lamp.
In segmentation, PointNet's errors on 426 and 212 follow height and position, as in Q2 and Q3: it labels the lower part of a deep seat as legs, and a seat that curves up into the backrest as back. DGCNN's labels follow the part boundaries on both chairs. Points on the same surface have similar neighbourhoods and so get similar edge features, which fits this result. Chairs with unusual geometry or annotation stay hard for both models. On 235, a sofa with a skirt down to the floor, PointNet predicts legs in the lower region and DGCNN labels half of the arm points as seat. On 351 both predict a base where the ground truth has back and arms.
DGCNN costs more to run. Its neighbour search is quadratic in the number of points (12.8 GB per layer at 10000 points and batch 32, as computed above), so I trained and evaluated it on 1024 or 2048 points rather than 10000.