16-825 Assignment 4: Multi-View Transformers

Farnaz Alam Ahmed

1. Multi-View Data Preprocessing (20 points)

Each training sample is built from 8 views of one ScanNet++ scene. Each view is a 400×300 RGB image with depth. It is center-cropped to 300×300 and resized to 224×224 (Q1.1). The depth is unprojected to 3D points (Q1.2). Everything is expressed in the first camera's frame and normalized in scale (Q1.3). Finally, a ray (Q1.4) and its Plücker coordinates (Q1.5) are computed for every pixel.

1.2 Depth unprojection (original world frame, meters)

Each pixel \((u, v)\) with z-depth \(Z\) is unprojected as \(\mathbf{X}_\text{world} = R\,(Z\,K^{-1}[u, v, 1]^\top) + \mathbf{t}\), using the resized intrinsics and the camera-to-world pose. Points from all 8 views fuse into a single consistent scene, and each camera frustum sits where its view was captured.

Unprojected points and cameras, bedroom scene
Scene 079a326597 (bedroom): points and cameras in the original world frame (m).
Unprojected points and cameras, office scene
Scene 07ff1c45bb (office): points and cameras in the original world frame (m).

1.3 Canonicalized geometry (first-camera frame, normalized units)

After canonicalization, camera 0 is the identity, so its center is at the origin. The valid-point centroid lies exactly 1 unit from it. In the screenshots below, the red diamond marks the first camera center and the yellow diamond marks the valid-point centroid. The black line between them is labeled with the centroid distance, and the highlighted box on the right repeats both values from the visualization page. The point cloud is slightly faded so that the markers stay visible.

Canonicalized points and cameras, bedroom scene
Scene 079a326597 (bedroom): first camera center (0.000, 0.000, 0.000); valid-point centroid distance 1.000; scene scale 2.080 m.
Canonicalized points and cameras, office scene
Scene 07ff1c45bb (office): first camera center (0.000, 0.000, 0.000); valid-point centroid distance 1.000; scene scale 1.966 m.

Canonicalization summary for the 5 visualized scenes

SampleFirst camera center (X, Y, Z)Valid-point centroid distanceScene scale (m)
036bce3393_frame_004470(0.0000, 0.0000, 0.0000)1.0001.867
076c822ecc_frame_002890(0.0000, 0.0000, 0.0000)1.0002.830
079a326597_frame_007430(0.0000, 0.0000, 0.0000)1.0002.080
07f5b601ee_frame_003630(0.0000, 0.0000, 0.0000)1.0000.924
07ff1c45bb_frame_000460(0.0000, 0.0000, 0.0000)1.0001.966

The scene scale is the centroid distance in meters before normalization. It is the divisor applied to translations, depth and points.

1.7 Final data check

SampleValid depthMax point reproj. error (px)Max ray reproj. error (px)Centroid distance
e050c15a8d_frame_00708097.8%1.10 × 10⁻⁴9.39 × 10⁻⁵1.0000
5f99900f09_frame_00282099.0%1.11 × 10⁻⁴1.11 × 10⁻⁴1.0000
47b37eb6f9_frame_00272096.1%9.59 × 10⁻⁵8.90 × 10⁻⁵1.0000
4c5c60fa76_frame_00669075.2%1.09 × 10⁻⁴1.01 × 10⁻⁴1.0000

All checks pass. Every reprojection error is about 10⁻⁴ px, roughly 100× below the 0.01 px requirement. What remains is float32 rounding. The centroid distance is 1 for every sample. All batch shapes match the data contract (batch 2, 8 views, 224×224): rgb, points_world and the rays are [2, 8, 3, 224, 224], plucker is [2, 8, 6, 224, 224], K is [2, 8, 3, 3] and T_world_from_camera is [2, 8, 4, 4].

Full check_data.py output
{
  "split": "train",
  "dataset_samples": 224,
  "checked_samples": 4,
  "samples": [
    {
      "sequence_name": "e050c15a8d_frame_007080",
      "views": 8,
      "resolution": 224,
      "valid_depth_fraction": 0.9779127240180969,
      "max_reprojection_error_px": 0.00011009880108758807,
      "max_ray_reprojection_error_px": 9.394258086103946e-05,
      "centroid_distance": 1.0
    },
    {
      "sequence_name": "5f99900f09_frame_002820",
      "views": 8,
      "resolution": 224,
      "valid_depth_fraction": 0.9903116226196289,
      "max_reprojection_error_px": 0.00011108566104667261,
      "max_ray_reprojection_error_px": 0.00011108566104667261,
      "centroid_distance": 0.9999999403953552
    },
    {
      "sequence_name": "47b37eb6f9_frame_002720",
      "views": 8,
      "resolution": 224,
      "valid_depth_fraction": 0.9609947800636292,
      "max_reprojection_error_px": 9.591897105565295e-05,
      "max_ray_reprojection_error_px": 8.897326188161969e-05,
      "centroid_distance": 1.0
    },
    {
      "sequence_name": "4c5c60fa76_frame_006690",
      "views": 8,
      "resolution": 224,
      "valid_depth_fraction": 0.7516716122627258,
      "max_reprojection_error_px": 0.0001089862416847609,
      "max_ray_reprojection_error_px": 0.00010141761595150456,
      "centroid_distance": 0.9999999403953552
    }
  ],
  "batch_shapes": {
    "rgb": [2, 8, 3, 224, 224],
    "depth": [2, 8, 1, 224, 224],
    "depth_valid": [2, 8, 1, 224, 224],
    "K": [2, 8, 3, 3],
    "frame_numbers": [2, 8],
    "T_world_from_camera": [2, 8, 4, 4],
    "original_world_from_reference": [2, 4, 4],
    "scene_scale": [2],
    "points_world": [2, 8, 3, 224, 224],
    "ray_origins": [2, 8, 3, 224, 224],
    "ray_directions": [2, 8, 3, 224, 224],
    "plucker": [2, 8, 6, 224, 224]
  }
}

Explanations

Canonicalization

Raw ScanNet++ poses live in an arbitrary per-scan world frame, and each room has its own physical size. A model would otherwise have to infer an absolute position and scale it cannot observe. Canonicalization removes both. First, every camera-to-world pose is re-expressed relative to the first sampled camera: \(T'_i = T_0^{-1}\,T_i\), computed in float64. This makes camera 0 the identity, at the origin and looking down \(+Z\). Depth is then unprojected in this frame, and the valid-point centroid distance \(s = \lVert \operatorname{mean}(\mathbf{p}_\text{valid}) \rVert\) is computed. Translations, depth and points are all divided by \(s\), and rotations are left unchanged. The centroid is not subtracted, so camera 0 stays at the origin and the scene's centroid sits exactly 1 unit away. Every training sample therefore shares the same reference frame and scale.

World-space ray directions

For pixel \((u, v)\), \(K^{-1}[u, v, 1]^\top\) gives the viewing direction in camera coordinates, with \(z = 1\). Rotating it by the camera-to-world rotation and normalizing gives the world-space unit direction \(\mathbf{d} = \dfrac{R\,K^{-1}[u, v, 1]^\top}{\lVert R\,K^{-1}[u, v, 1]^\top \rVert}\). Only the rotation is applied: a direction says which way the pixel looks, not where the camera is. The ray origin \(\mathbf{o}\) is the camera center \(\mathbf{t}\), shared by every pixel of that view. Every surface point seen by the pixel lies on \(\mathbf{o} + \lambda\mathbf{d}\), \(\lambda \ge 0\). This is what the ray reprojection check verifies.

Plücker moment

The Plücker coordinates of a ray are \([\mathbf{d}, \mathbf{m}]\) with moment \(\mathbf{m} = \mathbf{o} \times \mathbf{d}\). The moment does not depend on which point of the line is used as the origin: \((\mathbf{o} + \lambda\mathbf{d}) \times \mathbf{d} = \mathbf{o} \times \mathbf{d}\), because \(\mathbf{d} \times \mathbf{d} = 0\). So the 6-vector describes the line itself, not one point on it. Geometrically, \(\mathbf{m}\) is normal to the plane containing the ray and the origin, and with unit \(\mathbf{d}\), \(\lVert \mathbf{m} \rVert\) is the ray's perpendicular distance from the origin. Camera 0 sits at the canonical origin, so all of its rays have \(\mathbf{m} = 0\). Other cameras' moments encode where they are relative to camera 0. This dense 6-channel map is the camera conditioning used by the models in Parts 2 and 3.

2. Novel-View Synthesis with Mini-LVSM (40 points)

2.1 Overfitting one batch

The model was trained for 2000 steps on a single fixed training batch, with the same model and optimizer settings as 2.2. It reached a final PSNR of 30.99 dB (SSIM 0.872, MSE 8.5 × 10⁻⁴) on that batch, which passes the ~30 dB sanity check. The averaged training PSNR over all 2000 steps was 25.40 dB (it starts near 12 dB). The run took 717 s on an NVIDIA A10G.

Overfit visualization: source views, ground truth and prediction
val_epoch000.png from the overfit run. Columns: the 2 source views, ground-truth target and prediction. The prediction reproduces the memorized target's layout and colors, with only slight blur at fine detail.

2.2 Training and evaluation

Training settings

Model14.6M parameters; patch size 8 (784 tokens per view); width 384; 8 layers; head dim 64; MLP ratio 4; QK-norm
DataScanNet++ iPhone, 224×224, 8 views per sample: 2 source views and 6 target views; 224 training clips
OptimizationAdamW, lr 2 × 10⁻⁴, betas (0.9, 0.95), weight decay 0.05; 200 warmup steps, then cosine decay to 10%; gradient clip 1.0
ScheduleBatch size 4, 100 epochs (56 steps per epoch, 5600 steps); bf16 autocast; seed 825
HardwareOne NVIDIA A10G (24 GB, AWS g5.xlarge); ~28 s per epoch, 46 min in total

Training curves

Training and validation loss, PSNR and SSIM over 100 epochs
MSE loss and PSNR for training and validation, and validation SSIM, per epoch. Validation uses 2 source views on the 28 validation clips (168 target images).

Validation PSNR rises from 12.2 dB to 16.13 dB and SSIM from 0.38 to 0.561; the best checkpoint is the final epoch. Train and validation curves stay close (final training PSNR 15.60 dB), so the model is not overfitting. Both curves are still rising slowly at epoch 100, so longer training would likely help further.

Training visualizations

Validation visualization after epoch 0
Epoch 0. Columns: 2 source views, ground truth, prediction.
Validation visualization after epoch 95
Epoch 95. Same layout and validation clips.

After epoch 0 the predictions are a nearly uniform gray. By epoch 95 they recover the coarse color layout of each held-out target: bright windows and walls, dark furniture and floor in roughly the right places. Object shapes, fine texture and thin structures are still very blurry and patch-blocky.

Ablations: number of source views × target-ray conditioning

The best checkpoint is evaluated on the validation split with 1, 2 or 4 source views, using the remaining views as targets. Target rays are either correct, shuffled (the rays of a different target view) or none (all zeros).

Source viewsCorrect raysShuffled raysNo rays (zeros)Target images
PSNR (dB)SSIMPSNR (dB)SSIMPSNR (dB)SSIM
115.190.55811.800.48812.480.527196
216.130.56111.490.47212.460.526168
416.890.57611.590.47312.470.536112

Evaluation visualizations

Columns: source views, ground truth, prediction. Each image shows the first validation clips.

1 source view, correct rays
1 source view, correct rays
2 source views, correct rays
2 source views, correct rays
4 source views, correct rays
4 source views, correct rays
2 source views, shuffled rays
2 source views, shuffled rays
2 source views, no rays
2 source views, no rays
Remaining ablation images
1 source view, shuffled rays
1 source view, shuffled rays
1 source view, no rays
1 source view, no rays
4 source views, shuffled rays
4 source views, shuffled rays
4 source views, no rays
4 source views, no rays

Discussion

Number of source views. With correct rays, quality improves steadily with more source views: 15.19 → 16.13 → 16.89 dB, although the model was trained only with 2. More views cover more of the scene, so less of the target has to be guessed. The model can use them because attention takes any number of source tokens, and the Plücker rays tell it where each source pixel sits relative to the target camera. The gain shrinks per added view (+0.94 dB, then +0.76 dB for two more), as views increasingly overlap.

Ray conditioning. The target rays are the only information about which view to render. With all-zero rays, PSNR falls to about 12.5 dB for every source count, and adding views no longer helps. The model then outputs a blurry average-looking image, which minimizes MSE when the viewpoint is unknown. Shuffled rays are worse still (about 11.5 dB, lowest SSIM). The model trusts the rays and renders a confident but wrong viewpoint, and a sharp wrong image costs more MSE than a blurry average one. Together these show that the model really does use the camera geometry, rather than memorizing scene appearance.

Limitations. Predictions are blurry and show 8×8 patch seams. Small model, 100 epochs on 224 clips, and the MSE loss all favor smooth averages of plausible outputs. Regions not visible in any source view cannot be recovered and come out as a smooth guess.

2.3 Bonus: training with a random number of source views

Question. The baseline always trains with S = 2 source views, yet it is evaluated with 1, 2 and 4. Does training with a random S ∈ {1, 2, 3, 4} at each step (remaining views as targets) make the model better across view counts, especially for S = 1 and S = 4, which the baseline never saw?

Setup. A controlled comparison: identical model, optimizer, schedule, 100 epochs, data and seed (825). The only change is that each training step draws S uniformly from {1, 2, 3, 4} (config configs/lvsm_randS.json, key train_num_source_views). Validation during training still uses S = 2, so the best checkpoint is chosen the same way. Both checkpoints are evaluated with the same 9-setting ablation.

Validation PSNR curves and evaluation PSNR for the baseline and random-S model
Left: validation PSNR (S = 2) during training. Right: evaluation PSNR with correct rays, by number of source views. The bar axis starts at 14 dB to make the small differences visible.
Evaluation settingBaseline (S = 2) PSNR / SSIMRandom S PSNR / SSIMΔ PSNR (dB)
S = 1, correct15.19 / 0.55815.21 / 0.559+0.01
S = 2, correct16.13 / 0.56116.19 / 0.559+0.06
S = 4, correct16.89 / 0.57617.00 / 0.573+0.12
S = 1, shuffled11.80 / 0.48811.79 / 0.487−0.01
S = 2, shuffled11.49 / 0.47211.46 / 0.469−0.03
S = 4, shuffled11.59 / 0.47311.51 / 0.466−0.08
S = 1, none12.48 / 0.52712.46 / 0.532−0.02
S = 2, none12.46 / 0.52612.43 / 0.528−0.03
S = 4, none12.47 / 0.53612.46 / 0.537−0.01
Baseline, 4 source views
Baseline, 4 source views (correct rays)
Random-S model, 4 source views
Random-S model, 4 source views (correct rays)

Results. The two models are nearly identical. The validation curves overlap throughout training. The largest difference is +0.12 dB at S = 4, the setting the baseline never trained on, and S = 1 is unchanged. SSIM differs by under 0.01 everywhere, and the shuffled/no-ray settings are unchanged; the images are visually indistinguishable.

Interpretation. Training with varying S gives at most a small benefit for more views than trained with, and nothing for fewer. The baseline already generalizes across view counts. Attention is agnostic to sequence length, and every source token carries its own Plücker rays, so the per-token computation does not depend on how many other sources are present. The small model is capacity-limited (~16 dB), and that bottleneck dominates any effect of the view-count curriculum.

Limitations. One seed per configuration. Differences of ~0.1 dB are likely within run-to-run noise, so the S = 4 gain is only suggestive. A firmer answer needs several seeds, longer training, a larger model, and harder settings (for example, S up to 7 or wider baselines), where a fixed-S model might generalize less well.

3. Multi-View Geometry with Mini-VGGT (40 points)

3.1 Overfitting one scene

One fixed training clip (036bce3393_frame_004470, 8 views) was fitted for 40 epochs × 25 steps, with learning rate 10⁻³, on an NVIDIA A10G (~3 s per epoch).

Overfit loss curves: total, camera and depth
Overfit loss curves: total, camera (unweighted) and depth (unweighted).

All three losses fall steadily. The total loss ends below zero, which is expected: once predictions are accurate, the confidence term's −0.2·log(conf) reward outweighs the remaining weighted error.

Metric on the fitted clipEpoch 0Epoch 40
Total loss5.48−0.16
Camera loss / depth loss0.342 / 3.770.0027 / −0.175
Mean rotation error49.7°0.20°
Mean translation-direction error90.0°0.43°
Pose AUC @ 5° / 15° / 30°0 / 0 / 00.986 / 0.995 / 0.998
Depth AbsRel / δ10.619 / 0.2080.036 / 0.956
Ground-truth cameras and points for the overfit clip
Ground truth: canonical cameras (red = view 0) and depth-unprojected points.
Predicted cameras and points for the overfit clip
Prediction: points unprojected with the predicted cameras and depth.
Ground truth for the overfit clip seen from the camera viewpoint
Ground truth, seen from the cameras' viewpoint.
Prediction for the overfit clip seen from the camera viewpoint
Prediction, seen from the cameras' viewpoint.

The predicted geometry matches the ground truth closely: the camera trajectory, the room layout and the depth map all line up. The predicted first camera center is (0.0000, −0.0001, −0.0001), so the model has learned that view 0 is the identity. The main remaining difference is in the depth map, which shows faint 14×14 patch blocks from the per-patch decoder. It also fills in pixels that are missing (black) in the sensor depth, and those pixels are not supervised.

3.2 Training and evaluation

Training settings

ModelFrozen DINOv2 ViT-B/14; width 384, 6 heads, 8 frame/global pairs (16 blocks); camera head with 4 blocks; depth head with RGB refinement
DataScanNet++ iPhone, 8 views sampled from a 32-frame range, 224×224; 224 training clips, 28 validation clips
OptimizationAdamW, lr 6 × 10⁻⁴, weight decay 0.01; 10 warmup epochs, then cosine decay to 5%; loss weights camera 5, depth 1
ScheduleBatch size 16 clips (14 steps per epoch), 1000 epochs (default config); best checkpoint chosen by validation loss
HardwareOne NVIDIA A10G (24 GB, AWS g5.xlarge); ~15 GB GPU memory, 20–35 s per epoch including validation, ~9 h in total

Training curves

Training and validation loss curves: total, camera and depth
Total, camera (unweighted) and depth (unweighted) loss for training and validation over 1000 epochs.

Depth loss drops quickly in the first ~50 epochs and then flattens. Camera loss keeps decreasing slowly for the whole run. Validation loss is lower than training loss throughout. This is most likely because training clips use random view sampling within the 32-frame range, while validation uses a fixed, easier sampling of the first 32 prepared frames, not because of overfitting.

Validation and test metrics

Evaluated with best.pt: 8 views from the first 32 prepared frames, 224 px, seed 825, full splits (28 clips each, 784 view pairs).

SplitDepthRotation AUCTranslation AUCJoint pose AUC
AbsRel ↓δ1 ↑@5°@15°@30°@5°@15°@30°@5°@15°@30°
Validation0.1390.8340.1990.5180.7090.0130.0770.1960.0050.0610.180
Test0.1510.7970.2190.5220.7100.0090.0720.1830.0020.0480.163

Mean pairwise errors: rotation 11.8° (val) / 10.3° (test); translation direction 41.8° (val) / 40.5° (test). For comparison, the model at epoch 0 had depth AbsRel 0.73, and the overfit run reached 0.04 on its single clip.

Ground truth vs prediction on four held-out validation scenes

Each pair shows the GT (left) and the predicted cameras + depth (right) in the canonical first-camera frame. The viewpoint is behind the cameras, looking into the room, so the reconstructed surfaces face the viewer.

09c1414f1b: dining room

Scene 09c1414f1b ground truth
Ground truth
Scene 09c1414f1b prediction
Prediction

21d970d8de: office floor and chair

Scene 21d970d8de ground truth
Ground truth
Scene 21d970d8de prediction
Prediction

280b83fcf3: hallway

Scene 280b83fcf3 ground truth
Ground truth
Scene 280b83fcf3 prediction
Prediction
Scene 280b83fcf3 ground truth seen from the camera viewpoint
Ground truth, seen from the cameras' viewpoint
Scene 280b83fcf3 prediction seen from the camera viewpoint
Prediction, seen from the cameras' viewpoint

2b1dc6d6a5: office desk

Scene 2b1dc6d6a5 ground truth
Ground truth
Scene 2b1dc6d6a5 prediction
Prediction
Scene 2b1dc6d6a5 ground truth seen from the camera viewpoint
Ground truth, seen from the cameras' viewpoint
Scene 2b1dc6d6a5 prediction seen from the camera viewpoint
Prediction, seen from the cameras' viewpoint

Discussion of errors on held-out scenes

What works. On unseen rooms the model recovers the overall scene layout and scale. Floors, walls, the hallway and the desk area are in the right places, and the valid-point centroid distance of the predictions is 0.97–0.99 (GT: 1.00), so the predicted metric scale is nearly correct. Depth is good at the coarse level (δ1 ≈ 0.80–0.83, AbsRel ≈ 0.14–0.15), and relative rotations are mostly within 15–30° (rotation AUC@30 ≈ 0.71). Val and test are close, so the model generalizes consistently to new scenes.

Depth errors. Predicted depth is over-smoothed. Thin structures disappear (the chair legs and table edges in 09c1414f1b), depth discontinuities are blurred into ramps, and flat walls become slightly wavy and thick in 3D. The depth head decodes each 14×14 DINO patch with a linear layer plus a tiny convolutional refinement, so it cannot represent detail finer than a patch. The L1-style loss also favors smooth averages where it is uncertain.

Camera errors. Translation direction is the weakest output (AUC@30 ≈ 0.19, mean error ~41°), and it drags down the joint pose AUC. Views come from at most 32 consecutive video frames, so baselines are short and often dominated by rotation. For a short baseline, a small absolute error in translation becomes a large error in its direction. The model also has to infer translation purely from parallax in frozen DINO features, without iterative refinement. Rotation is easier: it follows directly from how image content shifts between views. Fine accuracy is low everywhere (AUC@5 ≤ 0.22), so the model gets cameras roughly right but not precisely.

Limitations. A small model (width 384) trained only on 224 clips for ~9 h on one A10G. The camera loss was still decreasing at the end of training, so longer training would likely improve poses further. Larger gains would probably come from a stronger depth decoder (for example, DPT-style multi-scale features), iterative camera refinement, more training data, or unfreezing DINO.

3.3 Bonus: how does the number of input views affect accuracy, runtime and memory?

Question. Mini-VGGT was trained only with 8 views, but its architecture accepts any number of views: frame attention is per view, and global attention simply sees a longer sequence. Does the trained model get better with more views (more parallax and more context), worse with fewer, or does it break outside its training setting? And how do runtime and memory scale?

Setup. Evaluation only, with the 3.2 checkpoint (best.pt, epoch 890) and no retraining. Full val and test splits, using the same protocol as 3.2 (first 32 prepared frames, 224 px, seed 825), with V ∈ {2, 4, 8, 12, 16, 24, 32}. The evaluation loader always takes frame 0 plus the first V−1 frames of a fixed per-scene random order. The view sets are therefore nested: every larger set contains the smaller ones, so V is the only thing that changes. As a check, V = 8 reproduces the 3.2 numbers exactly. Runtime and peak GPU memory were measured separately at batch size 1 on the A10G (bf16 autocast, 10 timed forward passes after warm-up; training step = forward + backward with frozen DINO).

Accuracy vs number of views

Depth AbsRel, depth delta1, rotation AUC@30 and translation AUC@30 vs number of views, for val and test
Accuracy of the 8-view-trained model as the number of input views changes (log-scale x-axis; dashed line = training setting).
ViewsPose pairsValidationTest
AbsRel ↓δ1 ↑Rot AUC@30Trans AUC@30AbsRel ↓δ1 ↑Rot AUC@30Trans AUC@30
2280.1090.8800.7190.2810.1140.8860.6990.210
41680.1290.8510.6700.2270.1390.8160.7270.212
8 (trained)7840.1390.8340.7090.1960.1510.7970.7090.183
1218480.1510.8060.6910.1720.1540.7950.7020.193
1633600.1630.8050.6720.1720.1590.7870.7140.191
2477280.1550.8000.6770.1690.1550.7990.7210.191
32138880.1640.7860.6630.1650.1570.8010.7240.189

Runtime and memory vs number of views

Time per sample and peak GPU memory vs number of views, for inference and a training step
Batch size 1 on an NVIDIA A10G. Inference: no_grad + bf16. Training step: forward + backward through everything except the frozen DINO.
ViewsTokens in global attentionInference (ms)Inference peak memory (GB)Training step (ms)Training peak memory (GB)
2522330.68780.90
41,044350.70791.09
82,088340.74781.48
164,176520.821092.28
246,264760.891623.09
328,3521060.972273.92

Results and conclusions

More views do not help. Depth is best with the fewest views: val AbsRel goes 0.109 → 0.139 → 0.164 for 2 → 8 → 32 views, with the same trend on test. Rotation AUC@30 stays at 0.66–0.73 with no clear trend, and translation AUC@30 falls on val (0.28 → 0.17) and is flat on test. The model does not use the extra parallax. It does degrade gracefully, though: even at 32 views (4× its training setting) no metric collapses, because attention with per-view 2D positions works for any number of views.

Possible reasons (not verified): extra views dilute global attention in a way the model never saw in training; with few views, the easier reference view 0 makes up a larger share of the depth metric; and pose AUCs at small V use few pairs, so they are noisy.

Cost. Inference time is flat (~34 ms) up to 8 views, then grows faster than linearly: 52 ms at 16 views, 106 ms at 32. Inference memory barely changes (0.7 → 1.0 GB). Training memory grows linearly, about 0.1 GB per view, so 32 views at batch size 16 would need about 60 GB, more than the 24 GB GPU.

Limitations. One checkpoint, trained at 8 views and evaluated only. A model trained with more or varying views might benefit from them. All views come from the same 32-frame window, so extra views add denser coverage, not wider baselines.