1. Multi-View Data Preprocessing
8 views per sample, 224 × 224 images. Intrinsics use the half-pixel resize convention. Depth is optical-axis z-depth.


Canonicalization: transform each pose by T₀⁻¹Tᵢ, then divide translations, depths and points by the distance to the valid-point centroid. The centroid is not subtracted, so camera 0 stays at the origin.
World-space rays: rotate K⁻¹[u,v,1] by the camera-to-world rotation and normalize. The origin is the camera center. Rays from every view therefore share one coordinate frame.
Plücker moment: m = o × d. The six-channel encoding [d,m] represents the ray line and is unchanged when o moves along d.
Data check
Four training samples pass: maximum point reprojection error 1.23e-04 px, maximum ray error 1.08e-04 px, and centroid distances approximately 1.
Full check_data.py output
{
"split": "train",
"dataset_samples": 224,
"checked_samples": 4,
"samples": [
{
"sequence_name": "e050c15a8d_frame_007080",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9779127240180969,
"max_reprojection_error_px": 0.00010844774078577757,
"max_ray_reprojection_error_px": 9.41436446737498e-05,
"centroid_distance": 1.0
},
{
"sequence_name": "5f99900f09_frame_002820",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9903116226196289,
"max_reprojection_error_px": 0.00012302029062993824,
"max_ray_reprojection_error_px": 0.0001078959321603179,
"centroid_distance": 1.0
},
{
"sequence_name": "47b37eb6f9_frame_002720",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9609947800636292,
"max_reprojection_error_px": 9.5367431640625e-05,
"max_ray_reprojection_error_px": 8.217109279939905e-05,
"centroid_distance": 1.0
},
{
"sequence_name": "4c5c60fa76_frame_006690",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.7516716122627258,
"max_reprojection_error_px": 0.00010783717152662575,
"max_ray_reprojection_error_px": 9.53696362557821e-05,
"centroid_distance": 1.0
}
],
"batch_shapes": {
"rgb": [
2,
8,
3,
224,
224
],
"depth": [
2,
8,
1,
224,
224
],
"depth_valid": [
2,
8,
1,
224,
224
],
"K": [
2,
8,
3,
3
],
"frame_numbers": [
2,
8
],
"T_world_from_camera": [
2,
8,
4,
4
],
"original_world_from_reference": [
2,
4,
4
],
"scene_scale": [
2
],
"points_world": [
2,
8,
3,
224,
224
],
"ray_origins": [
2,
8,
3,
224,
224
],
"ray_directions": [
2,
8,
3,
224,
224
],
"plucker": [
2,
8,
6,
224,
224
]
}
}2. Novel-View Synthesis with Mini-LVSM
2.1 Overfitting
One fixed sample, 8 views, 2 source views, 2000 updates. Final PSNR: 36.74 dB; SSIM: 0.9379.

2.2 Training and evaluation
224 training scenes, 100 epochs, batch 1, 8 views, 2 sources and 6 targets. Patch size 8, width 384, 8 transformer blocks. AdamW, learning rate 0.0002, 200 warm-up steps, cosine decay, weight decay 0.05, RGB MSE and BF16 autocast. Batch size was reduced from 4 to fit GPU memory. The checkpoint with highest validation PSNR is selected (epoch 99, counting from 1).




Validation: all 28 scenes, 8 views from the first 32 prepared frames, seed 825. All nine settings use the same checkpoint.
| Sources | Target rays | PSNR (dB) ↑ | SSIM ↑ |
|---|---|---|---|
| 1 | correct | 14.85 | 0.5531 |
| 1 | shuffled | 11.76 | 0.4916 |
| 1 | none | 12.20 | 0.5408 |
| 2 | correct | 15.47 | 0.5516 |
| 2 | shuffled | 11.52 | 0.4809 |
| 2 | none | 12.11 | 0.5362 |
| 4 | correct | 15.91 | 0.5631 |
| 4 | shuffled | 11.72 | 0.4870 |
| 4 | none | 12.08 | 0.5455 |
More source views improve PSNR: 14.85 → 15.47 → 15.91 dB. Correct target rays specify the desired viewpoint; shuffled rays give the wrong viewpoint and zero rays remove it. At 2 sources, shuffling drops PSNR to 11.52 dB and removing rays to 12.11 dB. Correct-ray predictions remain blurry and lose fine furniture details; the model has learned coarse layout but limited detail.
Evaluation visualizations
Each image shows source views, ground truth and prediction from left to right.
1 source views



2 source views



4 source views



3. Multi-View Geometry with Mini-VGGT
3.1 Overfitting
One fixed 8-view clip, 40 epochs × 25 updates, learning rate 0.001. Total loss decreases from 5.48 to -0.147. Final depth AbsRel: 0.0389; δ1: 0.9522. The confidence term −0.2 log(c) allows negative loss values.



3.2 Training and evaluation
Frozen DINOv2 ViT-B/14, width 384, 8 frame/global transformer pairs, 6 heads, and a 4-block camera transformer. 1000 epochs, batch 16, 8 views, sampling range 32, 224 × 224. AdamW, learning rate 0.0006, 10 warm-up epochs, cosine decay, weight decay 0.01, camera/depth loss weights 5/1, BF16. The checkpoint with lowest validation loss is selected (epoch 843). Both models trained on one Quadro RTX 6000.

Full validation and test splits: 28 scenes each, 8 views from the first 32 prepared frames, 224 px, seed 825. No batch limit. AUC values range from 0 to 1.
| Split | Depth AbsRel ↓ | Depth δ1 ↑ |
|---|---|---|
| val | 0.1385 | 0.8390 |
| test | 0.1523 | 0.7899 |
| Split | Metric | AUC@5° ↑ | AUC@15° ↑ | AUC@30° ↑ |
|---|---|---|---|---|
| val | Rotation | 0.1918 | 0.4899 | 0.6746 |
| val | Translation | 0.0128 | 0.0897 | 0.2105 |
| val | Joint pose | 0.0033 | 0.0664 | 0.1821 |
| test | Rotation | 0.1827 | 0.5177 | 0.7129 |
| test | Translation | 0.0219 | 0.0717 | 0.1648 |
| test | Joint pose | 0.0051 | 0.0525 | 0.1527 |
Depth recovers the main room layout, but thin furniture structures and boundaries are smoothed. Camera errors cause thicker surfaces and imperfect alignment when views are fused. Translation direction is the main pose weakness: joint AUC is much lower than rotation AUC. Small camera baselines can make translation direction sensitive to small absolute errors.
Four validation scenes: ground truth and prediction








3.3 Bonus: Number of input views
Question: how do accuracy, runtime and memory change when the same 8-view-trained model receives 2, 4 or 8 input views? The checkpoint, resolution, seed and 32-frame prefix are fixed. All 28 validation and 28 test scenes are evaluated at each view count, with batch 1 and no retraining.
| Split | Views | AbsRel ↓ | δ1 ↑ | Rotation AUC@30 ↑ | Translation AUC@30 ↑ | Joint AUC@30 ↑ |
|---|---|---|---|---|---|---|
| val | 2 | 0.1035 | 0.8866 | 0.7095 | 0.2333 | 0.2048 |
| test | 2 | 0.1171 | 0.8649 | 0.7357 | 0.1571 | 0.1274 |
| val | 4 | 0.1280 | 0.8551 | 0.6875 | 0.2062 | 0.1869 |
| test | 4 | 0.1395 | 0.8203 | 0.7240 | 0.1556 | 0.1440 |
| val | 8 | 0.1385 | 0.8390 | 0.6746 | 0.2104 | 0.1820 |
| test | 8 | 0.1523 | 0.7899 | 0.7129 | 0.1648 | 0.1528 |
| Views | Forward time (ms/scene) | Peak allocated GPU memory (GiB) |
|---|---|---|
| 2 | 45.7 | 0.713 |
| 4 | 71.8 | 0.722 |
| 8 | 124.1 | 0.808 |
Cost is measured on the first validation scene on the same Quadro RTX 6000: 3 warm-up passes, then the mean of 20 BF16 forward passes, with CUDA synchronization. Data loading and CPU-to-GPU transfer are excluded. Memory includes model weights, RGB input and forward allocations; it is allocated memory, not reserved memory.

Depth error increases with more views on both splits, while runtime and memory increase. More inputs therefore do not automatically improve this fixed model. Joint pose accuracy changes differently between validation and test, so there is no consistent pose improvement. The model was trained only at 8 views; changing the input count also changes its test-time context.
Limitation: each view count averages over different depth maps and pose pairs, and the target scene scale is recomputed from the selected views. These results compare the complete evaluation settings, not the isolated effect of extra context on identical targets. Only one checkpoint and one timing scene are tested; timing can vary with GPU load.



The same validation scene and reference RGB view with 2, 4 and 8 input views. Each panel uses its own view-dependent canonical scale and depth color range.