# Assignment 2: single-view 3D prediction

Open `index.html` for the report. `ASSIGNMENT.md` contains the course instructions.
The implementation covers Q1, Q2.1–Q2.5, and Q3.1 (implicit occupancy).

## Desktop environment

The experiments use `ssh desktop` and the RTX 5090. The desktop checkout is
`~/Desktop/Projects/L3D/proj2`; its dedicated conda environment is `l3d-proj2`.
The environment setup is recorded in `scripts/setup_desktop.sh`. It installs
Python 3.11, PyTorch 2.8.0 / torchvision 0.23.0 with CUDA 12.8, and builds
PyTorch3D for compute capability 12.0. `requirements-lock.txt`,
`pytorch3d-commit.txt`, and `environment.json` record the tested versions.

```sh
source ~/miniconda3/etc/profile.d/conda.sh
conda activate l3d-proj2
cd ~/Desktop/Projects/L3D/proj2
python -m pytest -q tests
```

For another Linux CUDA machine, create the environment using the setup script's
commands and set `TORCH_CUDA_ARCH_LIST` to that GPU's compute capability.
Set `CUDA_HOME` to the installed CUDA toolkit before building PyTorch3D.
The large CUDA dependencies are excluded from the submission ZIP.

## Dataset

Download the chair archive from the course's Hugging Face dataset:

```sh
mkdir -p data
curl -fL --retry 5 -C - -o data/r2n2_shapenet_dataset.zip \
  https://huggingface.co/datasets/learning3dvision/r2n2_shapenet_dataset/resolve/main/r2n2_shapenet_dataset.zip
unzip -q data/r2n2_shapenet_dataset.zip -d data
python prepare_data.py
```

Use `--data_root /path/to/data` or `L3D_DATA_ROOT` for another location.
Preparation requires CUDA: it uses the GPU to sample mesh surface points.
It caches geometry, all available RGB views, voxel targets, and a 10,000-point
surface pool. It retains the supplied test split and holds out 10% of training
objects for validation by a deterministic hash of object IDs. Training samples
an RGB view on each access. Validation and testing use the lowest available view.
The same three test chairs are rendered for every representation and ablation.

The dataset and preprocessing cache are not included in the ZIP.

## Fitting and training

```sh
python fit_data.py --type vox
python fit_data.py --type point
python fit_data.py --type mesh

python train_model.py --type vox
python train_model.py --type point
python train_model.py --type mesh
python train_model.py --type mesh --w_smooth 0 --run_dir runs/mesh_smooth0
python train_model.py --type mesh --w_smooth 1 --run_dir runs/mesh_smooth1
python train_model.py --type implicit
```

All full runs use the actual RGB encoder; `--load_feat` is disabled. Defaults:
seed 7, Adam at 0.0004, batch size 32, maximum 100,000 steps. Validation and
checkpoint saving run every 2,000 steps. After 10,000 steps, five consecutive
validation checks without improvement stop the run. `best.pth` is the best
validation checkpoint; `last.pth` also includes Adam state, random states, epoch,
and batch position. Training loss and validation metrics are in `history.jsonl`.
Point and mesh training draw target subsets from the cached 10,000-point surface
pool. Mesh predictions are sampled afresh each step. Evaluation always samples
the original ground-truth mesh.

```sh
python train_model.py --type point --load_checkpoint
```

The ZIP contains inference checkpoints only. A fresh training run writes new
resume checkpoints. To retain the supplied best checkpoint, use another
`--run_dir`, then pass that directory to evaluation.

Mesh training starts from a radius-0.35, level-4 icosphere. Chamfer uses squared
nearest-neighbor distances in both directions. Uniform Laplacian smoothing has
weight 0.1 in the baseline. Fixed mesh connectivity and the uniform Laplacian
operator are cached; the cached smoothing value and gradient are tested against
PyTorch3D's original calculation. Direct voxel fitting uses learning rate 0.05 for the
convex optimization, while every neural decoder uses 0.0004.

The implicit decoder trains on 2,048 uniformly sampled grid coordinates per
step. Evaluation queries the entire 32³ grid in chunks of 2,048. Both occupancy
models output logits; extraction applies sigmoid and threshold 0.5.

## Evaluate and render

```sh
python eval_model.py --type vox --load_checkpoint
python eval_model.py --type point --load_checkpoint
python eval_model.py --type mesh --load_checkpoint
python eval_model.py --type mesh --run_dir runs/mesh_smooth0 --load_checkpoint
python eval_model.py --type mesh --run_dir runs/mesh_smooth1 --load_checkpoint
python eval_model.py --type implicit --load_checkpoint
python interpret.py --type point --run_dir runs/point
python build_report.py
python verify_artifacts.py
```

Evaluation loads `runs/<experiment>/best.pth` and restores its architecture
settings. `--checkpoint` can select another checkpoint. Each run evaluates the
complete test split, samples 1,000 surface points with fixed per-object seeds,
and records percentages at thresholds 0.01 through 0.05. The starter's voxel
rotation and centering conventions are preserved. Empty voxel surfaces receive
zero precision, recall, and F1. Point and mesh coordinates are not recentered.

The renderer uses matched 256 × 256 views, 32 frames per turntable, and looping
GIFs at approximately 15 fps. A PNG of the first frame accompanies every GIF.
Occlusion replaces one of 16 input patches with the ImageNet mean color and
measures the change in point predictions using squared Chamfer distance.

`python smoke_gpu.py` verifies a 300-update overfit on two chairs using the
actual RGB encoder. `tests/test_core.py` checks losses and gradients, decoder
shapes, variable mesh batch sizes, empty outputs, deterministic evaluation,
and checkpoint resume against uninterrupted training.

## Report and submission

The report uses relative local links and opens offline. To preview it:

```sh
python -m http.server 8000
```

`python package_submission.py` creates `jjacob2_code_proj2.zip` in the parent
folder. It includes source, tests, reproduction instructions, best inference
checkpoints, recorded metrics, and report assets. It excludes datasets,
environments, Git metadata, caches, setup logs, and intermediate checkpoints.
Publishing the webpage and uploading the ZIP are separate steps.

## Sources

Starter code and data: [learning3d/assignment2](https://github.com/learning3d/assignment2),
commit `7bb80cba23f59b87a01f489fcf373e74cd97fedb`.
PyTorch3D provides rendering, surface sampling, nearest-neighbor searches, and
uniform Laplacian smoothing. Its license is included as `LICENSE-PyTorch3D`.
