lnayak
Depth is optical-axis z-depth. I unproject each pixel with the inverse intrinsics and transform it with the camera-to-world pose. Canonicalization expresses all cameras and points in camera 0's frame, then divides translations, depth and points by the valid-point centroid distance. Camera 0 stays at the origin; the centroid is not subtracted.
Ray directions are rotated into the world frame and normalized. The Plücker encoding is [d, o × d], where o is the camera center and d is the unit world-space ray direction. The moment o × d records the ray's offset from the origin and stays the same when o moves along the ray. Both stages below use the same five training scenes and frame selections.










{
"split": "train",
"dataset_samples": 224,
"checked_samples": 4,
"samples": [
{
"sequence_name": "e050c15a8d_frame_007080",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9779127240180969,
"max_reprojection_error_px": 0.00011141905270051211,
"max_ray_reprojection_error_px": 0.00010764275066321716,
"centroid_distance": 1.0
},
{
"sequence_name": "5f99900f09_frame_002820",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9903116226196289,
"max_reprojection_error_px": 0.00012302029062993824,
"max_ray_reprojection_error_px": 0.00011108566104667261,
"centroid_distance": 1.0
},
{
"sequence_name": "47b37eb6f9_frame_002720",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9609947800636292,
"max_reprojection_error_px": 0.00010235906665911898,
"max_ray_reprojection_error_px": 8.897326188161969e-05,
"centroid_distance": 1.0
},
{
"sequence_name": "4c5c60fa76_frame_006690",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.7516716122627258,
"max_reprojection_error_px": 0.00010924567322945222,
"max_ray_reprojection_error_px": 0.00010257320536766201,
"centroid_distance": 1.0
}
],
"batch_shapes": {
"rgb": [
2,
8,
3,
224,
224
],
"depth": [
2,
8,
1,
224,
224
],
"depth_valid": [
2,
8,
1,
224,
224
],
"K": [
2,
8,
3,
3
],
"frame_numbers": [
2,
8
],
"T_world_from_camera": [
2,
8,
4,
4
],
"original_world_from_reference": [
2,
4,
4
],
"scene_scale": [
2
],
"points_world": [
2,
8,
3,
224,
224
],
"ray_origins": [
2,
8,
3,
224,
224
],
"ray_directions": [
2,
8,
3,
224,
224
],
"plucker": [
2,
8,
6,
224,
224
]
}
}After 2,000 updates on one fixed batch, final overfit PSNR was 30.97 dB. This reached the assignment's approximate 30 dB target.

The overfit predictions reproduce the table, chairs, wall colors and target viewpoints. Edges are softer than GT, and a grid pattern remains visible across the wall and furniture.
| Training setting | Value |
|---|---|
| Training | 300 epochs (100 initial + 200 continued) |
| Optimizer | AdamW, learning rate 0.0002, weight decay 0.05 |
| Batch size | 4 |
| Views / sampling window | 2 source + 6 target views / 32 frames |
| Resolution | 224 × 224 |
| Transformer | Width 384, 8 layers |
| Patch size | 8 |
| Precision | BF16 autocast on CUDA |
| Warmup | 200 updates |
| Seed | 825 |
Both LVSM models used 100 initial epochs followed by 200 continued epochs. The provided resume path restores the model and optimizer, extends the cosine schedule to 300 epochs and advances it to the saved step. The same continuation procedure was used for both models.
At 100 epochs, baseline training and validation PSNR were 15.61 and 16.14 dB. At 300 epochs they were 16.22 and 16.81 dB. The blurry initial predictions motivated the longer run.


| Source views | Target rays | PSNR (dB) | SSIM |
|---|---|---|---|
| 1 | correct | 15.6989 | 0.5776 |
| 1 | shuffled | 11.6828 | 0.4818 |
| 1 | none | 12.4323 | 0.5269 |
| 2 | correct | 16.8361 | 0.5833 |
| 2 | shuffled | 11.2916 | 0.4612 |
| 2 | none | 12.4277 | 0.5254 |
| 4 | correct | 17.8988 | 0.6007 |
| 4 | shuffled | 11.3172 | 0.4577 |
| 4 | none | 12.4581 | 0.5362 |









Correct rays recover coarse window, wall and furniture placement, but predictions remain blurry and show a patch grid. Four source views make the bright lamp region clearer in this scene, while thin chair legs and shelf details are still missing. Shuffled rays retain the scene's broad appearance but place it in the wrong target view. Zero rays produce the same blurry image for every target: with fixed sources, all target-query inputs are then identical.
With correct target rays, PSNR for 1, 2 and 4 source views was 15.70, 16.84 and 17.90 dB. The 4-view minus 1-view difference was +2.20 dB. The target set changes with the source count, so this comparison also changes which images are scored.
At 1 source views, correct rays changed PSNR by +4.02 dB relative to shuffled rays and +3.27 dB relative to zero rays. At 2 source views, correct rays changed PSNR by +5.54 dB relative to shuffled rays and +4.41 dB relative to zero rays. At 4 source views, correct rays changed PSNR by +6.58 dB relative to shuffled rays and +5.44 dB relative to zero rays.
How does reducing transformer depth affect novel-view synthesis? I compare the baseline and smaller model after 300 epochs using the same validation source counts and correct target rays.
| Training setting | Value |
|---|---|
| Training | 300 epochs (100 initial + 200 continued) |
| Optimizer | AdamW, learning rate 0.0002, weight decay 0.05 |
| Batch size | 4 |
| Views / sampling window | 2 source + 6 target views / 32 frames |
| Resolution | 224 × 224 |
| Transformer | Width 384, 4 layers |
| Patch size | 8 |
| Precision | BF16 autocast on CUDA |
| Warmup | 200 updates |
| Seed | 825 |
| Changed setting | Baseline | Small model |
|---|---|---|
| Transformer | Width 384, 8 layers | Width 384, 4 layers |
| Model | Trainable parameters | Forward ms / scene | Peak allocated MiB | Training + validation hours |
|---|---|---|---|---|
| baseline | 14633408 | 50.3325 | 439.3691 | 1.1451 |
| small | 7539136 | 25.0872 | 412.6621 | 0.7701 |
Timing used NVIDIA RTX 6000 Ada Generation, FP32; autocast and TF32 disabled; eval mode; no_grad, one shared scene, two source views and six target views at 224 pixels. Each measurement used five warmup passes and 20 timed forwards. Forward time excludes loading, transfers and metrics. Peak allocated memory includes model, inputs and output; logged epoch time includes training and validation.
The smaller model took 0.498 times the baseline forward time and changed peak allocated memory by -26.7 MiB. This timing covers one scene on one device.
| Source views | Baseline PSNR | Small PSNR | Difference (dB) | Baseline SSIM | Small SSIM |
|---|---|---|---|---|---|
| 1 | 15.6989 | 15.6055 | -0.0934 | 0.5776 | 0.5741 |
| 2 | 16.8361 | 16.7377 | -0.0984 | 0.5833 | 0.5802 |
| 4 | 17.8988 | 17.6036 | -0.2952 | 0.6007 | 0.5973 |




The smaller model captures broad window, wall and floor colors. With four source views, the bright lamp region is more visible in some predictions, but chair legs, shelf contents and textures remain blurred. A patch grid is still visible. More source images do not resolve the missing fine detail in this example.
The smaller model changed PSNR by -0.09, -0.10, -0.30 dB at 1, 2 and 4 source views. It scored higher in 0 of the three settings. Reduced capacity may limit reconstruction detail, while fewer layers can also change optimization. A single run per model cannot separate these effects or establish variation across random seeds.
Does changing the reconstruction loss improve image quality? These experiments all use 100 epochs, the 8-layer model, AdamW at 0.0002, batch size 4 and seed 825. The baseline in these tables is the saved 100-epoch MSE run. The 300-epoch model above is not used as the control here. Checkpoints are selected by validation PSNR.
The perceptual term averages L1 feature differences at VGG16 relu1_2, relu2_2, relu3_3 and relu4_3. VGG16 is pretrained on ImageNet and stays frozen. All reported image losses are MSE, regardless of the training objective.
| Training loss | Source views | PSNR | SSIM |
|---|---|---|---|
| MSE | 1 | 15.1928 | 0.5583 |
| MSE | 2 | 16.1446 | 0.5605 |
| MSE | 4 | 16.8779 | 0.5749 |
| MSE + 0.1 L1 | 1 | 15.2289 | 0.5584 |
| MSE + 0.1 L1 | 2 | 16.2458 | 0.5622 |
| MSE + 0.1 L1 | 4 | 16.9908 | 0.5781 |
| MSE + 0.2 (1 - SSIM) | 1 | 14.7850 | 0.5764 |
| MSE + 0.2 (1 - SSIM) | 2 | 15.4411 | 0.5799 |
| MSE + 0.2 (1 - SSIM) | 4 | 15.8396 | 0.5935 |
| MSE + 0.05 perceptual | 1 | 14.6876 | 0.5648 |
| MSE + 0.05 perceptual | 2 | 15.2934 | 0.5650 |
| MSE + 0.05 perceptual | 4 | 15.6071 | 0.5763 |

At two source views, the changes from MSE were MSE + 0.1 L1: +0.10 dB PSNR and +0.0016 SSIM. MSE + 0.2 (1 - SSIM): -0.70 dB PSNR and +0.0194 SSIM. MSE + 0.05 perceptual: -0.85 dB PSNR and +0.0045 SSIM.
| Training loss | Predicted edge magnitude | GT edge magnitude | Edge error |
|---|---|---|---|
| MSE | 0.0060 | 0.0182 | 0.0208 |
| MSE + 0.1 L1 | 0.0062 | 0.0182 | 0.0209 |
| MSE + 0.2 (1 - SSIM) | 0.0028 | 0.0182 | 0.0187 |
| MSE + 0.05 perceptual | 0.0055 | 0.0182 | 0.0202 |
Edge magnitude averages absolute horizontal and vertical RGB differences over every target image. Edge error measures the difference from the GT image gradients. The table quantifies changes in sharpness, while the matched images show their appearance. Higher edge magnitude can reflect detail or patch-grid artifacts, so it is not an overall quality score. This tests one weight per auxiliary loss and one random seed.
I compare two fixed source views with a source count sampled uniformly from 1 to 4 at each training step. Each batch still has eight views; the remaining views are targets. The sampled counts are saved with the run and can be resumed.
| Training source count | Updates |
|---|---|
| 1 | 1448 |
| 2 | 1361 |
| 3 | 1391 |
| 4 | 1400 |
| Source views | Baseline PSNR | Random sources PSNR | Baseline SSIM | Random sources SSIM |
|---|---|---|---|---|
| 1 | 15.1928 | 15.2330 | 0.5583 | 0.5585 |
| 2 | 16.1446 | 16.1753 | 0.5605 | 0.5593 |
| 4 | 16.8779 | 17.0195 | 0.5749 | 0.5740 |
Random-source training changed PSNR by +0.04, +0.03, +0.14 dB at 1, 2 and 4 evaluation sources. It exposes the model to all tested source counts, but also changes the number of supervised targets per update. This experiment does not separate those effects.
I replace the six Plücker channels with 16 camera values: the 3 by 3 camera-to-world rotation, three normalized translations and fx/W, fy/H, cx/W, cy/H. These values are repeated across pixels before patch projection. Source tokens still contain RGB, while target tokens contain only the camera encoding. The transformer and training schedule are unchanged; the input projections have more parameters.
| Source views | Baseline PSNR | Raw cameras PSNR | Baseline SSIM | Raw cameras SSIM |
|---|---|---|---|---|
| 1 | 15.1928 | 15.1517 | 0.5583 | 0.5582 |
| 2 | 16.1446 | 15.9933 | 0.5605 | 0.5575 |
| 4 | 16.8779 | 16.6826 | 0.5749 | 0.5708 |
Raw camera parameters changed PSNR by -0.04, -0.15, -0.20 dB at 1, 2 and 4 sources. Unlike Plücker rays, the repeated camera vector does not explicitly describe each pixel's viewing ray. The network must learn that mapping using the positional embedding. This comparison also changes projection size.


| Training setting | Value |
|---|---|
| Training | 40 epochs, 1,000 updates |
| Optimizer | AdamW, learning rate 0.001, weight decay 0.01 |
| Batch size | 1 per update from 1 fixed clip |
| Views / sampling window | 8 views / 32 frames |
| Resolution | 224 × 224 |
| Transformer | Width 384, 8 frame/global pairs (16 blocks) |
| Patch size | 14 |
| Precision | BF16 autocast |
| Warmup | 0 epochs |
| Seed | 825 |
| Encoder | Frozen DINOv2 ViT-B/14 |
| Loss weights | Camera 5, depth 1 |
| GPUs | 1 |


The overfit room layout and camera path closely match GT. Some surfaces appear thicker, with scattered points around the upper wall edges.
Overfit total loss changed from 5.4830 to -0.1688. The render uses the best saved checkpoint at epoch 40; this fixed training clip does not measure generalization.
The depth objective includes a negative log-confidence term, so depth and total loss can fall below zero.
| Training setting | Value |
|---|---|
| Training | 1,000 epochs, 14,000 updates |
| Optimizer | AdamW, learning rate 0.0006, weight decay 0.01 |
| Batch size | 16 total (4 per GPU) |
| Views / sampling window | 8 views / 32 frames |
| Resolution | 224 × 224 |
| Transformer | Width 384, 8 frame/global pairs (16 blocks) |
| Patch size | 14 |
| Precision | BF16 autocast |
| Warmup | 10 epochs |
| Seed | 825 |
| Encoder | Frozen DINOv2 ViT-B/14 |
| Loss weights | Camera 5, depth 1 |
| GPUs | 4 |

Evaluation uses eight views from the first 32 prepared frames, 224-pixel images and seed 825, across all 28 validation and 28 test scenes. The table reports AUC in [0, 1]. Translation and joint-pose AUC exclude pairs with undefined ground-truth translation direction.
| Metric | Validation | Test |
|---|---|---|
| Depth AbsRel | 0.1364 | 0.1655 |
| Depth δ1 | 0.8292 | 0.7710 |
| Rotation AUC 5° | 0.1944 | 0.1809 |
| Rotation AUC 15° | 0.5183 | 0.4929 |
| Rotation AUC 30° | 0.7072 | 0.7116 |
| Translation AUC 5° | 0.0115 | 0.0156 |
| Translation AUC 15° | 0.0844 | 0.0759 |
| Translation AUC 30° | 0.1946 | 0.1844 |
| Joint pose AUC 5° | 0.0023 | 0.0038 |
| Joint pose AUC 15° | 0.0640 | 0.0506 |
| Joint pose AUC 30° | 0.1692 | 0.1663 |
Validation depth AbsRel was 0.1364, compared with 0.1655 on test. Depth δ1 was 0.8292 and 0.7710. These depth and pose errors can both affect the reconstructed cloud. The four examples below show individual scenes and do not represent the full error distribution.
Translation-direction accuracy is the main pose weakness: validation AUC at 15 degrees is 0.0844, versus 0.5183 for rotation. Camera errors can misalign points from different views even when individual depth maps are reasonable. The held-out clouds retain broad structure, but are less consistent than the overfit example.
GT and predicted geometry use the same camera, bounds and GT-valid pixels in each comparison. Predictions are neither aligned nor rescaled to fit GT; bounds include outliers.




I evaluate the same checkpoint with 2, 4, 8, 16 and 32 input views on both complete held-out splits. The frame selections are nested prefixes. Common-view metrics always score the first two views and their camera pair. Both predictions and GT receive the dataset-derived conversion to the two-view canonical scale; no alignment is fitted. Aggregate scores include every input view and pair, so their evaluated content changes with view count.
| Common-view metric | 2 inputs | 4 inputs | 8 inputs | 16 inputs | 32 inputs |
|---|---|---|---|---|---|
| Depth AbsRel | 0.1243 | 0.1333 | 0.1320 | 0.1712 | 0.1859 |
| Depth δ1 | 0.8193 | 0.8304 | 0.8330 | 0.8024 | 0.7932 |
| Rotation AUC 5° | 0.2714 | 0.1857 | 0.1857 | 0.1857 | 0.1643 |
| Rotation AUC 15° | 0.5833 | 0.5500 | 0.5500 | 0.5095 | 0.4738 |
| Rotation AUC 30° | 0.7238 | 0.7143 | 0.7310 | 0.6988 | 0.6738 |
| Translation AUC 5° | 0.0000 | 0.0000 | 0.0071 | 0.0071 | 0.0071 |
| Translation AUC 15° | 0.1286 | 0.0929 | 0.1048 | 0.1214 | 0.1262 |
| Translation AUC 30° | 0.2952 | 0.2226 | 0.2548 | 0.2512 | 0.2702 |
| Joint pose AUC 5° | 0.0000 | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| Joint pose AUC 15° | 0.1119 | 0.0643 | 0.0738 | 0.0762 | 0.0667 |
| Joint pose AUC 30° | 0.2619 | 0.1905 | 0.2214 | 0.2155 | 0.2214 |
| Input views | Aggregate AbsRel | Aggregate δ1 | Aggregate joint AUC 30° | Forward ms | Peak allocated MiB |
|---|---|---|---|---|---|
| 2 | 0.1243 | 0.8193 | 0.2619 | 29.5512 | 706.6514 |
| 4 | 0.1313 | 0.8372 | 0.1587 | 27.1984 | 724.3076 |
| 8 | 0.1364 | 0.8291 | 0.1697 | 34.2974 | 762.5625 |
| 16 | 0.1753 | 0.7912 | 0.1394 | 30.5714 | 842.0952 |
| 32 | 0.1877 | 0.7693 | 0.1349 | 36.9486 | 986.7690 |
On val, using 32 instead of 8 inputs changed common-view AbsRel by +0.0539, δ1 by -0.0397 and joint-pose AUC at 30 degrees by -0.0000.
| Common-view metric | 2 inputs | 4 inputs | 8 inputs | 16 inputs | 32 inputs |
|---|---|---|---|---|---|
| Depth AbsRel | 0.1195 | 0.1353 | 0.1569 | 0.1677 | 0.1737 |
| Depth δ1 | 0.8580 | 0.8370 | 0.7758 | 0.7488 | 0.7467 |
| Rotation AUC 5° | 0.2500 | 0.2500 | 0.2357 | 0.2571 | 0.2500 |
| Rotation AUC 15° | 0.4905 | 0.5119 | 0.5405 | 0.5095 | 0.5167 |
| Rotation AUC 30° | 0.6595 | 0.7012 | 0.7083 | 0.6988 | 0.7000 |
| Translation AUC 5° | 0.0000 | 0.0071 | 0.0429 | 0.0000 | 0.0071 |
| Translation AUC 15° | 0.0452 | 0.1190 | 0.1214 | 0.0881 | 0.0857 |
| Translation AUC 30° | 0.1512 | 0.2512 | 0.2512 | 0.2619 | 0.2190 |
| Joint pose AUC 5° | 0.0000 | 0.0071 | 0.0214 | 0.0000 | 0.0000 |
| Joint pose AUC 15° | 0.0095 | 0.0429 | 0.0738 | 0.0357 | 0.0476 |
| Joint pose AUC 30° | 0.1036 | 0.1821 | 0.1905 | 0.2012 | 0.1679 |
| Input views | Aggregate AbsRel | Aggregate δ1 | Aggregate joint AUC 30° | Forward ms | Peak allocated MiB |
|---|---|---|---|---|---|
| 2 | 0.1195 | 0.8580 | 0.1036 | 26.8002 | 706.6514 |
| 4 | 0.1472 | 0.8158 | 0.1748 | 27.9886 | 724.3076 |
| 8 | 0.1655 | 0.7710 | 0.1657 | 34.1045 | 762.5625 |
| 16 | 0.1806 | 0.7530 | 0.1572 | 28.2735 | 842.0952 |
| 32 | 0.1872 | 0.7486 | 0.1574 | 37.0324 | 986.7690 |
On test, using 32 instead of 8 inputs changed common-view AbsRel by +0.0169, δ1 by -0.0291 and joint-pose AUC at 30 degrees by -0.0226.





Timing used NVIDIA RTX 6000 Ada Generation, bf16 autocast and batch size 1, with five warmup passes and 20 measured passes. It includes the encoder and prediction heads, but excludes loading, transfers, metrics and rendering. Memory is peak allocated CUDA memory, including the model and inputs. Each split's timing uses its first scene.
The checkpoint was trained with eight views. Other counts change its input distribution as well as the available context. Extra views need not improve every output, and these single-scene timings do not measure uncertainty or general throughput.
I compare direct XYZ prediction with reconstruction from cameras and depth in the same joint model at epoch 350. The point head projects patch tokens into signed XYZ maps and adds an RGB-guided convolutional residual. Its loss is mean absolute XYZ error on GT-valid pixels, using the supplied 98 percent trimming. Camera, depth and point weights are 5, 1 and 1.
This is an intermediate-checkpoint study: 350 epochs from the original 1,000-epoch schedule, not a completed 1,000-epoch run or a 350-epoch cosine schedule. Training uses frozen DINOv2 ViT-B/14, eight frame/global pairs of width 384, eight views from a 32-frame window, global batch size 16, AdamW at 0.0006 and seed 825.
| Split | Reconstruction | XYZ L1 | Point L2 | Point L2 (m) |
|---|---|---|---|---|
| val | Camera + depth | 0.1566 | 0.3228 | 0.6586 |
| val | Direct points | 0.1539 | 0.3168 | 0.6464 |
| test | Camera + depth | 0.1524 | 0.3051 | 0.5953 |
| test | Direct points | 0.1532 | 0.3047 | 0.5931 |
Evaluation uses the same GT-valid pixels without trimming, confidence filtering or fitted alignment. L1 averages absolute XYZ error; L2 is Euclidean distance. Conversion to meters uses the dataset's scene scale.
On val, direct prediction changed mean point distance by -0.0121 m relative to camera/depth reconstruction. Lower is better.
On test, direct prediction changed mean point distance by -0.0023 m relative to camera/depth reconstruction. Lower is better.

Direct prediction avoids combining separate camera and depth errors, but does not enforce ray consistency. These intermediate checkpoints show current behavior, not the final performance after convergence.
Does joint supervision help camera estimation? I compare camera-only training with joint camera/depth/point training at exactly 350 epochs. Both use the same architecture, seed, batch size, sampling and learning-rate schedule. The inactive depth head is frozen in the camera-only model. I compare camera metrics only, using fixed-epoch checkpoints rather than separate best-checkpoint selection.
| Split | Supervision | Rotation AUC 5° | Rotation AUC 15° | Rotation AUC 30° | Translation AUC 5° | Translation AUC 15° | Translation AUC 30° | Joint pose AUC 5° | Joint pose AUC 15° | Joint pose AUC 30° |
|---|---|---|---|---|---|---|---|---|---|---|
| val | Camera only | 0.1505 | 0.4209 | 0.6053 | 0.0059 | 0.0433 | 0.1105 | 0.0038 | 0.0314 | 0.0948 |
| val | Camera + depth + points | 0.1426 | 0.4173 | 0.6139 | 0.0036 | 0.0412 | 0.1192 | 0.0010 | 0.0296 | 0.1042 |
| test | Camera only | 0.1413 | 0.4317 | 0.6432 | 0.0026 | 0.0322 | 0.1148 | 0.0010 | 0.0202 | 0.0958 |
| test | Camera + depth + points | 0.1579 | 0.4463 | 0.6588 | 0.0056 | 0.0339 | 0.1136 | 0.0010 | 0.0163 | 0.0934 |
On val, joint supervision changed rotation AUC at 15 degrees by -0.0037 and joint-pose AUC at 30 degrees by +0.0094.
On test, joint supervision changed rotation AUC at 15 degrees by +0.0145 and joint-pose AUC at 30 degrees by -0.0025.

This tests transfer to the camera task. It does not establish an effect on depth-only or point-only training. Shared geometry supervision may help camera features, while competing losses can also hurt them. The comparison uses one seed; runs used RTX A6000 and RTX 6000 Ada GPUs, so numerical differences may also contribute.
Both models start from the same baseline checkpoint, selected at epoch 781. Each receives 100 additional epochs with a fresh AdamW optimizer, learning rate 0.00002, five warmup epochs, global batch size 16 and the original camera/depth loss. One keeps DINO frozen; the other updates its used parameters. This is a paired fine-tuning experiment, not a comparison of training the encoder from scratch.
| Split | Encoder | Depth AbsRel | Depth δ1 | Rotation AUC 5° | Rotation AUC 15° | Rotation AUC 30° | Translation AUC 5° | Translation AUC 15° | Translation AUC 30° | Joint pose AUC 5° | Joint pose AUC 15° | Joint pose AUC 30° |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| val | Frozen | 0.1374 | 0.8277 | 0.1929 | 0.5180 | 0.7075 | 0.0112 | 0.0844 | 0.1938 | 0.0023 | 0.0639 | 0.1686 |
| val | Unfrozen | 0.1463 | 0.8165 | 0.1860 | 0.5248 | 0.7077 | 0.0171 | 0.0895 | 0.2086 | 0.0048 | 0.0724 | 0.1886 |
| test | Frozen | 0.1669 | 0.7708 | 0.1814 | 0.4936 | 0.7118 | 0.0158 | 0.0760 | 0.1840 | 0.0038 | 0.0503 | 0.1656 |
| test | Unfrozen | 0.1733 | 0.7816 | 0.1837 | 0.5173 | 0.7216 | 0.0084 | 0.0574 | 0.1637 | 0.0000 | 0.0298 | 0.1426 |
On val, unfreezing changed AbsRel by +0.0089, δ1 by -0.0112 and joint-pose AUC at 30 degrees by +0.0201.
On test, unfreezing changed AbsRel by +0.0064, δ1 by +0.0108 and joint-pose AUC at 30 degrees by -0.0230.


The saved frozen encoder is unchanged from initialization, while the unfrozen encoder has changed weights. Updating the encoder can adapt image features to this data, but can also overfit a small training set. These results depend on the chosen starting checkpoint and fine-tuning rate.