1. Multi-View Data Preprocessing
RGB, depth, and camera parameters are cropped and resized together to 224 × 224. The implemented geometry places points, cameras, and rays in a consistent coordinate frame.
1.2 Unprojection
Each depth pixel becomes a 3D point through z K⁻¹[u, v, 1], followed by its camera-to-world transform. The viewer below shows the resulting colored point cloud and camera frustums in the original world frame.
Open unprojection viewer in a full window ↗
1.3 Canonicalization
The first sampled camera defines the reference frame. Camera translations, depth, and points are scaled so the valid-point centroid is one unit from the origin. Camera 0 is at (0, 0, 0); the centroid is not translated to the origin.
Open canonicalization viewer in a full window ↗
1.1, 1.4–1.6 Implementation notes
Intrinsics. Resizing uses the pixel-center mapping u′ = sx(u + 0.5) − 0.5 and the corresponding vertical transform after cropping.
Rays. A ray starts at the camera center and points along normalized R K⁻¹[u, v, 1]. Its Plücker representation is [d, o × d]; the moment does not change when the origin moves along that ray.
Sampling. Training randomly samples views within a temporal window. Validation and test use deterministic prepared samples and the provided evaluation geometry.
1.7 Final geometry check
| Sequence | Views | Valid depth | Point error (px) | Ray error (px) | Centroid distance |
|---|---|---|---|---|---|
| e050c15a8d / 007080 | 8 | 97.8% | 0.0001106 | 0.0000959 | 1.000000 |
| 5f99900f09 / 002820 | 8 | 99.0% | 0.0001221 | 0.0001079 | 1.000000 |
| 47b37eb6f9 / 002720 | 8 | 96.1% | 0.0000954 | 0.0000864 | 1.000000 |
| 4c5c60fa76 / 006690 | 8 | 75.2% | 0.0001141 | 0.0000942 | 1.000000 |
All four checked clips have reprojection errors below 0.00013 pixels. Full numerical results.
2. Novel-View Synthesis with Mini-LVSM
2.1 Overfitting
After 2,000 steps on a fixed training batch, PSNR reaches 30.90 dB and SSIM 0.870.

2.2 Training and evaluation
Settings: 8 transformer layers, width 384, 8 × 8 patches, 2 source views during training, batch size 4, 100 epochs. AdamW with learning rate 2 × 10⁻⁴, 200 warmup steps, cosine decay, and weight decay 0.05.

Source views and target-ray conditioning
| Source views | Target rays | PSNR (dB) ↑ | SSIM ↑ |
|---|---|---|---|
| 1 | correct | 15.119 | 0.5557 |
| 1 | shuffled | 11.767 | 0.4853 |
| 1 | none | 12.495 | 0.5318 |
| 2 | correct | 16.109 | 0.5590 |
| 2 | shuffled | 11.445 | 0.4709 |
| 2 | none | 12.463 | 0.5292 |
| 4 | correct | 16.832 | 0.5734 |
| 4 | shuffled | 11.569 | 0.4742 |
| 4 | none | 12.482 | 0.5388 |
Each image below shows source views on the left, followed by the ground-truth target and prediction. Click an image for its full resolution.
1 source view
2 source views
4 source views
Correct ray conditioning gives substantially higher PSNR than shuffled or absent rays. Additional source views improve the measured score, although the target-image counts differ across settings (196, 168, and 112), so the comparison does not hold the target population fixed.
The held-out predictions recover coarse color and lighting, but remain blurry. Fine furniture edges and scene detail are not reconstructed well, despite the strong fixed-batch overfit result.
2.3 Optional experiment: MSE + L1 loss
Question: does adding absolute-error supervision improve novel-view synthesis? The variant uses L = MSE + 0.1 L1. Architecture, initialization seed, optimizer, sampling, and 100-epoch budget are held fixed. Both checkpoints are selected by validation PSNR and evaluated using the same PSNR/SSIM definitions.
| Source views | MSE PSNR | MSE + L1 PSNR | Δ PSNR | MSE SSIM | MSE + L1 SSIM |
|---|---|---|---|---|---|
| 1 | 15.119 | 15.174 | +0.055 | 0.5557 | 0.5574 |
| 2 | 16.109 | 16.216 | +0.106 | 0.5590 | 0.5611 |
| 4 | 16.832 | 16.937 | +0.105 | 0.5734 | 0.5760 |
The L1 variant yields small gains with correct rays: about 0.11 dB for both two and four source views. The images still show substantial blur; this single-seed result does not establish a large perceptual improvement.
All L1-variant conditioning results
| Source views | Target rays | PSNR (dB) ↑ | SSIM ↑ |
|---|---|---|---|
| 1 | correct | 15.174 | 0.5574 |
| 1 | shuffled | 11.730 | 0.4844 |
| 1 | none | 12.496 | 0.5320 |
| 2 | correct | 16.216 | 0.5611 |
| 2 | shuffled | 11.385 | 0.4693 |
| 2 | none | 12.471 | 0.5286 |
| 4 | correct | 16.937 | 0.5760 |
| 4 | shuffled | 11.481 | 0.4711 |
| 4 | none | 12.506 | 0.5372 |
3. Multi-View Geometry with Mini-VGGT
3.1 Overfitting
One fixed training scene, 40 epochs, and 1,000 optimizer steps. Final depth AbsRel is 0.0363, depth δ₁ is 95.6%, and rotation error is 0.24°.


3.2 Training and evaluation
Settings: frozen DINOv2 ViT-B/14 encoder; 8 frame/global attention pairs, width 384, 6 heads; 8 views, sampling window 32, batch size 16. Trained for 1,000 epochs with AdamW, learning rate 6 × 10⁻⁴, 10-epoch warmup, cosine decay, and weight decay 0.01. Camera and depth loss weights are 5 and 1.
Training completed all 14,000 optimizer steps. The checkpoint at epoch 849 achieved the lowest validation loss (0.8583) and is used for every final evaluation below.

Held-out results
| Metric | Validation | Test |
|---|---|---|
| Depth AbsRel ↓ | 0.1455 | 0.1542 |
| Depth δ₁ ↑ | 0.8300 | 0.7895 |
| Rotation error (°) ↓ | 13.8403 | 9.4372 |
| Translation-direction error (°) ↓ | 42.6551 | 42.7558 |
| Rotation AUC @5° ↑ | 0.1640 | 0.2051 |
| Rotation AUC @15° ↑ | 0.4624 | 0.5279 |
| Rotation AUC @30° ↑ | 0.6627 | 0.7238 |
| Translation AUC @5° ↑ | 0.0143 | 0.0105 |
| Translation AUC @15° ↑ | 0.0735 | 0.0543 |
| Translation AUC @30° ↑ | 0.1778 | 0.1615 |
| Joint pose AUC @5° ↑ | 0.0028 | 0.0036 |
| Joint pose AUC @15° ↑ | 0.0493 | 0.0412 |
| Joint pose AUC @30° ↑ | 0.1479 | 0.1493 |
28 validation scenes and 28 test scenes. Eight views sampled from the first 32 prepared frames at 224 × 224, seed 825. AUC and δ₁ values are fractions; higher is better.
Depth δ₁ reaches 83.0% on validation and 79.0% on test. Test depth error is slightly higher, while test rotation error is lower. Translation direction remains the weaker pose component: mean errors exceed 42° on both splits, and joint pose AUC is much lower than rotation AUC.
Four held-out scenes
Ground truth and predictions use the same coordinate frame, axis limits, and depth color scale. These are final best-checkpoint results.




Open final vggt geometry viewer in a full window ↗
3.3 Optional experiment: Number of input views
Question: how do additional input views affect geometry accuracy and inference cost? The same eight-view-trained checkpoint is evaluated with nested subsets of 2, 4, and 8 images. Scene scale stays fixed. Depth is compared on the same reference-view pixels, and pose on the same camera pair (0, 1).
| Input views | Reference AbsRel ↓ | Reference δ₁ ↑ | Rotation error (°) ↓ | Translation error (°) ↓ | Forward ms/scene ↓ | Peak GPU GiB ↓ |
|---|---|---|---|---|---|---|
| 2 | 0.1917 | 68.3% | 15.59 | 37.44 | 3.54 | 1.07 |
| 4 | 0.1549 | 79.4% | 13.84 | 33.89 | 4.55 | 1.36 |
| 8 | 0.1287 | 87.2% | 13.17 | 35.01 | 8.47 | 1.96 |
All settings use 28 scenes and the same 1,281,355 valid reference-view pixels. Forward times are batched throughput per scene, with CUDA synchronization and warm-up excluded; data loading and metric calculation are excluded. Peak allocated GPU memory includes the model and shared input batch.



Reference-view depth accuracy increases from 68.3% to 79.4% to 87.2% as views increase from 2 to 4 to 8. Eight views cost about 2.4× the forward time and 1.8× the peak memory of two views. Rotation error improves modestly; translation error is lowest with four views, so the pose benefit is not uniform.
This measures inference-time sensitivity of a model trained with eight views. It does not compare models trained separately at each view count.



















