Assignment 2: Single-view 3D reconstruction¶
All models use a pretrained ResNet18 encoder trained with RGB inputs on CUDA; provided image features were not used (load_feat=False). Training used 100,000 steps, batch size 32, and Adam with learning rate 0.0004. Baseline results use the last periodic checkpoint at step 98,000; Q3 results use step 99,999.
Scores are mean per-object F1@0.05 (%) over 678 chair test objects, with 1,000 surface points and fixed evaluation seed 825. The displayed examples are test indices 0, 226, and 452. Blue denotes predictions and gray denotes ground truth; each panel includes the input RGB and ground-truth mesh.
2.1. Image to voxel grid¶
The decoder maps the 512 image features to a 64 × 4 × 4 × 4 feature volume, then uses trilinear upsampling and 3D convolutions to reach 32³ occupancy logits. Training uses mean binary cross entropy after sigmoid; surfaces are extracted at probability 0.5.
F1@0.05: 77.140%. The examples recover the main chair shapes, but edges and thin legs are rounded or thickened at this resolution.




2.2. Image to point cloud¶
An MLP 512 → 512 → 1024 → 3000 predicts 1,000 XYZ points, trained with symmetric mean squared-distance Chamfer loss. The output has no fixed connectivity, allowing points to cover separate legs and the chair back.
F1@0.05: 76.537%. The thin-legged chairs are recognizable, while the armchair's surfaces remain sparsely represented by the finite point set.




2.3. Image to mesh¶
An MLP predicts XYZ offsets for the 2,562 vertices of an ico_sphere(4). The loss combines Chamfer distance between sampled surface points with uniform Laplacian smoothing (weights 1.0 and 0.1).
F1@0.05: 73.499%. The outputs form continuous surfaces, but the examples show tapered legs and distorted armchair geometry.




| Representation | Mean F1@0.05 (%) |
|---|---|
| Voxels | 77.140 |
| Point cloud | 76.537 |
| Mesh | 73.499 |
In these runs, voxels exceed points by 0.603 percentage points and meshes by 3.641 points. Dense occupancy supervision and local 3D convolutions plausibly help capture overall shape. Points have flexible coverage but no surface connectivity. Deforming a fixed sphere with smoothing can make thin legs and gaps difficult to represent. These are interpretations of this experiment, not a universal ranking of the representations.
2.4. Effect of the voxel occupancy threshold¶
I varied the inference occupancy threshold from 0.3 to 0.7, holding the checkpoint, test objects, input views, sampling seed, and point count fixed. No retraining was needed. This probability threshold is distinct from the 0.05 distance threshold used for F1.
| Occupancy threshold | Precision@0.05 (%) | Recall@0.05 (%) | F1@0.05 (%) | Empty predictions |
|---|---|---|---|---|
| 0.3 | 67.335 | 80.879 | 72.811 | 0 |
| 0.4 | 72.578 | 81.361 | 76.103 | 0 |
| 0.5 | 76.342 | 79.432 | 77.140 | 1 |
| 0.6 | 78.115 | 75.178 | 75.590 | 2 |
| 0.7 | 78.145 | 68.474 | 71.585 | 9 |
0.5 gives the best mean F1 among the tested values. Lower thresholds expand the predicted surface and increase false positives. Higher thresholds remove uncertain geometry: precision increases, but recall drops and more predictions become empty. Thus, increased confidence does not necessarily improve reconstruction.
The same chair at 0.3 and 0.7 illustrates thicker versus thinner geometry. This particular example improves at 0.7 even though the dataset mean decreases, so a single visualization would be misleading for choosing the threshold. This sweep is a test-set sensitivity analysis, not independent validation of an optimal threshold.


2.5. Interpretation: consistency across input views¶
For each chair, I reconstructed three different input views independently with the same voxel checkpoint. The GIFs rotate all predictions and the ground-truth mesh together using a shared center, scale, and camera. The input photographs remain fixed; there is no retraining or multi-view fusion.
Chair 0: the armchair shape persists, but back height and armrest proportions change across views. Mean pairwise occupancy IoU: 0.453.

Chair 226: all three predictions retain the back, seat, and legs, with changes in thickness and proportions. Mean pairwise occupancy IoU: 0.359.

Chair 452: input view 6 produces a fragmented seat and legs, while view 8 produces a much thicker chair. Mean pairwise occupancy IoU: 0.272.

The model learns recurring chair structure, while uncertain geometry remains view-dependent. Pairwise IoU measures agreement between predicted occupied voxels, not accuracy: consistent predictions can still be wrong. These three objects illustrate behavior rather than establish a test-set-wide consistency score.
3.1. Implicit network¶
An image-conditioned MLP maps (512 image features, x, y, z) → occupancy logit with hidden widths 256, 256, and 128. Training uses BCE on 4,096 randomly sampled grid locations per image. Evaluation queries the full 32 × 32 × 32 grid over normalized coordinates from −1 to 1 and extracts the 0.5 surface.
F1@0.05: 76.115%, 1.025 percentage points below the standard voxel decoder. The armchair and straight-backed chair are recognizable, but chair 452 has fragmented thin structures. A continuous occupancy function does not automatically improve detail when evaluated on the same coarse grid; the convolutional decoder also has an explicit local spatial structure. Although these three examples score better individually than their voxel baselines, the full-test mean is lower.




3.2. Parametric network¶
Four independent MLP patches map (512 image features, u, v) → (x, y, z), using hidden widths 256, 256, and 128. Each patch receives 250 samples from the square [−1, 1]², producing 1,000 points in total. Training uses fresh UV samples and fresh ground-truth surface samples with Chamfer loss; evaluation fixes the UV samples.
F1@0.05: 73.989%, 2.548 percentage points below the direct point decoder. Chair 226 is reconstructed well, but the armchair and rounded-back chair are less accurate. The shared patch functions constrain point placement more than independently predicting all coordinates, which may limit coverage of complex geometry in this configuration.



