psalmon
All 3D figures are PyTorch3D renders: meshes and voxel surfaces with the mesh renderer and a point light at the camera, point clouds with the point renderer, as in Assignment 1. Animated figures rotate every shape through 360 degrees about the vertical axis. Voxel grids are drawn as their marching-cubes surface at occupancy 0.5, in the same world coordinates as the meshes.
voxel_loss is binary cross entropy between predicted occupancy probabilities and the target grid. Each voxel is optimized through a logit and sigmoid. After 1,000 steps, the loss fell from 0.693 to 0.021, and the fitted grid matches the target.
The provided R2N2 voxel files are rotated 90 degrees about the up axis relative to the ShapeNet meshes. The dataset loader now maps binvox (x, y, z) to mesh (z, y, -x) before aligning bounding boxes. With that fix, surfaces of the ground truth voxels score 94.9% F1@0.05 against their own meshes. All voxel and implicit results below use these corrected labels.
chamfer_loss finds nearest neighbors with knn_points and averages squared nearest-neighbor distances in both directions. The 1,000-point fit reduced the loss from 0.382 to 0.00015.
smoothness_loss is the uniform Laplacian smoothing penalty. The mesh starts from a simple seat, back, and four legs, then fits 1,000 sampled surface points with Chamfer loss plus smoothing at weight 0.001. The 1,000-step objective fell from 0.01834 to 0.00073. It matches the chair's seat, back, and legs, though the legs come out slightly thin and slanted.
I used --load_feat: every model decodes the supplied pretrained ResNet18 features, to fit within local GPU memory. Each decoder trains on all 6,100 training chairs with batch size 32, sampling a random view from each chair's 10 training views at every step. Point, parametric, and mesh models compare against 1,000 target points drawn fresh at every step from 5,000 cached surface samples per chair. Evaluation uses all 678 held out chairs.
F1 follows the starter's compute_sampling_metrics at distance threshold 0.05. It samples 1,000 points (the starter's n_points) from each predicted surface and from each ground truth mesh; point and parametric decoders are scored on the 1,000 points they output. Hyperparameters were chosen on 300 held out training chairs, never on the test set.
A 3D transposed convolution decoder maps the 512 dimensional image feature to 32 cubed occupancy probabilities. Only about 5% of cells are occupied. With plain voxel_loss, the model under-fills: its validation recall was 42% and thin chairs came out nearly empty. I therefore weighted occupied cells more heavily in the binary cross entropy. On the validation chairs, F1@0.05 was 49.4%, 59.7%, 55.5%, and 46.2% for weights 1, 2, 3, and 5, so the final model uses weight 2.
Test F1@0.05: 60.41% (precision 59.8%, recall 62.8%). Solid chairs such as example 2 come out complete, with seat, back, and legs. For the thin metal chair in example 1, the model recovers the seat and back but breaks up the legs, and the boxy armchair in example 3 comes out as a lumpy box.


eval_vox.png).A multilayer perceptron with three hidden layers of width 1,024 predicts 1,000 XYZ points and trains with symmetric Chamfer loss for 40,000 steps with a cosine learning rate schedule. Test F1@0.05: 74.40% (precision 79.2%, recall 72.9%). The points show the seat, back, and legs of ordinary chairs, as in examples 1 and 2, but they are noisy: uncertain regions get a diffuse spread of points rather than a clean surface. The boxy armchair in example 3 comes out as a box-shaped cloud.


eval_point.png).As in the starter, the decoder deforms a level-4 ico-sphere (2,562 vertices): a multilayer perceptron with three hidden layers of width 1,024 predicts an offset for every vertex. The loss is Chamfer distance between 1,000 points sampled from the predicted and target surfaces, plus three regularizers:
smoothness_loss at weight 1;The sphere starts at radius 0.5 to match the size of the chairs. Chamfer plus Laplacian smoothing alone, at the starter's default weight 0.1, scored higher (70.96% on the test set) but produced crumpled, self-intersecting surfaces with spikes for legs. Section 2.4 shows how each regularizer trades F1 for a clean surface. I report the regularized model because its output is a usable mesh.
Test F1@0.05: 63.49% (precision 62.0%, recall 66.5%). Every example is a smooth, closed chair with a seat, back, and four legs, and the armchair keeps its boxy body. The price is detail. A single genus-0 surface cannot open the gaps between legs or behind a spindle back, so it bridges them with webbing, and the smoothing rounds the legs into tapered cones.
I also tried another initial mesh: my six-box chair template from Section 1.3, with the same decoder and Laplacian smoothing at weight 0.1. It scored 57.8% F1@0.05 on the validation chairs, against 71.4% for the ico-sphere under the same loss, and 57.9% on the test set. The back and all four legs collapsed to single points, and the seat box stretched into a wedge covering the whole chair. Each box is a separate eight-vertex component, so the uniform Laplacian is minimized by shrinking it to a point. Thin legs receive few Chamfer samples to resist that pull, and a collapsed box has zero area, so it is never sampled again and cannot recover. The template only kept its parts at weight 0.001 in Section 1.3. The ico-sphere is one connected surface with evenly spaced vertices, so it has no parts that can vanish.


eval_mesh.png).Comparison. Parametric (77.82%) and point cloud (74.40%) score highest, then implicit (65.02%), mesh (63.49%), and voxels (60.41%). Free points can go anywhere, so the point decoders put output only where chairs have surface; their precision is the highest, 79 to 80%. The mesh must stay one closed sheet deformed from a sphere, so it spends surface on webbing between legs and cannot form thin parts. I also regularized it for a clean surface, which costs about 7 points: with smoothing alone it scored 70.96%, between point clouds and implicit. The voxel decoder must get every one of 32,768 cells right, and a cell next to a thin part is ambiguous. Resolution is not the main limit: the ground truth 32 cubed grids themselves score 94.9%. The voxel model's trouble is class imbalance. Unweighted, it drops thin parts; weighted too heavily, it thickens them, and its best setting still loses on both precision and recall.
Part 1: Laplacian smoothing weight.
I varied the Laplacian smoothing weight of the ico-sphere mesh decoder over 0, 0.01, 0.1, 1, and 10, holding everything else fixed (20,000 steps each). Each model trained on 5,800 chairs and was scored on the 300 held out training chairs and on the test set.
| Smoothing weight | 0 | 0.01 | 0.1 | 1 | 10 |
|---|---|---|---|---|---|
| Validation F1@0.05 (%) | 71.7 | 70.9 | 71.4 | 67.7 | 54.4 |
| Test F1@0.05 (%) | 72.7 | 71.0 | 70.6 | 67.7 | 54.0 |


F1 is flat up to weight 0.1. The differences among 0, 0.01, and 0.1 are about one point, within the spread between validation and test. Above 0.1 it falls quickly: 67.7% at weight 1 and 54% at weight 10. The renders show why. With little smoothing, vertices move freely toward the target, so the surface reaches thin legs but folds into sharp creases. At weight 1 the seat and back become smooth sheets, but the few vertices still pulled toward the legs form long spikes. At weight 10 the Laplacian dominates: the shape stays close to a smooth, inflated sphere, and only a ring of spikes hints at the chair. The smoothing term never repairs the folds, because they come from stretching a sphere over a chair with holes. It trades detail for smoothness, and this F1 rewards detail up to about weight 0.1.
Part 2: regularizers for a clean surface. Smoothing alone never removes the folds, so I added PyTorch3D's edge-length and normal-consistency losses and a half-size starting sphere. I trained each setting for 10,000 steps on 5,800 chairs and scored it on the 300 held out chairs; the figure renders two of them.
| Laplacian | Edge | Normal | Sphere radius | Validation F1@0.05 (%) | Surface |
|---|---|---|---|---|---|
| 0.1 | 0 | 0 | 1 | 71.9 | crumpled folds, spiky legs |
| 0.1 | 0 | 0 | 0.5 | 72.2 | crumpled folds, spiky legs |
| 0.1 | 1 | 0.01 | 0.5 | 69.1 | smooth seat and back, spiky legs |
| 0.1 | 1 | 0.1 | 0.5 | 65.2 | smoother, some spikes |
| 0.5 | 1 | 0.05 | 0.5 | 64.8 | smooth but flame-like spikes |
| 1 | 1 | 0.01 | 0.5 | 64.8 | smooth, pointed legs |
| 1 | 1 | 0.1 | 0.5 | 62.3 | clean, smooth chair |

Shrinking the starting sphere changes nothing on its own. Adding the edge and normal-consistency losses is what turns the crumpled seat and back into smooth panels. Raising the normal weight to 0.1 or the Laplacian to 1 then shortens the leg spikes, but only both together remove them. The clean chair costs about 10 points of validation F1, and both precision and recall fall. That drop is real, but it mostly measures thin detail: the folds and spikes that the regularizers remove likely placed surface samples near legs and spindles, and F1 counts those samples as hits. So on this task F1 alone cannot choose the mesh: the weakly regularized models score best and look worst. Section 2.3 reports the most regularized setting, trained for the full 20,000 steps on all chairs, where it scores 63.49% on the test set.
I interpolated linearly between the pretrained features of two held out chairs, a spindle-back side chair (example 2) and a boxy armchair (example 3), and decoded each mixture with the point decoder. The output changes gradually. The tall back and separate legs of the side chair widen and fill in toward the armchair's box. The decoder therefore responds smoothly to the feature rather than switching between memorized shapes.

I also decoded four widely spaced camera views of the same chair. For example 2, all four predictions are the same recognizable side chair even though the viewpoint changes a lot. To measure this over many chairs, I averaged the pairwise Chamfer distance between the four view predictions over 40 test chairs: 0.00217. Predictions for different chairs differ by 0.00573, and the ground truth shapes of different chairs by 0.01715. So predictions for one chair vary about 2.6 times less across views than predictions for different chairs do: the model mostly encodes the object, not the viewpoint. But its predictions vary about 3 times less between chairs than real chairs do: it pulls every chair toward an average shape, which is why unusual chairs like the armchair come out vague.

The implicit decoder takes a 3D coordinate from -1 to 1 on each axis and a 256 dimensional projection of the image feature, then predicts occupancy. Sine and cosine coordinate features help it model thin parts. Training samples occupied, nearby empty, and random empty cells; evaluation queries all 32 cubed locations and extracts the 0.5 surface. Test F1@0.05: 65.02% (precision 62.1%, recall 70.9%), above the voxel decoder's 60.41% on the same grid. It recovers a back, seat, and separate legs for examples 1 and 2, and a box with sides for the armchair. Because it trains on balanced occupied and empty samples, it avoids the voxel decoder's choice between dropping and inflating thin parts, though its surfaces are thicker than the true ones.


The parametric decoder takes a regular 2D grid of 1,000 UV samples and a 256 dimensional projection of the image feature. It maps four patches of samples to XYZ, each through its own multilayer perceptron with three hidden layers of width 512, and trains with symmetric Chamfer loss for 40,000 steps. Test F1@0.05: 77.82% (precision 80.1%, recall 78.0%), the best of all models. The patches cover the seat and back of ordinary chairs well, but they are continuous maps of a square, so they spread across thin legs instead of forming them, and the armchair comes out as a diffuse box.

