Q1: Multi-View Data Preprocessing
Depth Unprojection & Canonicalization
Scene 1



Scene 2



Scene 3



Scene 4



Scene 5



Final Check
{
"split": "train",
"dataset_samples": 224,
"checked_samples": 4,
"samples": [
{
"sequence_name": "e050c15a8d_frame_007080",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9779127240180969,
"max_reprojection_error_px": 0.00011009880108758807,
"max_ray_reprojection_error_px": 9.394258086103946e-05,
"centroid_distance": 1.0
},
{
"sequence_name": "5f99900f09_frame_002820",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9903116226196289,
"max_reprojection_error_px": 0.00011108566104667261,
"max_ray_reprojection_error_px": 0.00011108566104667261,
"centroid_distance": 0.9999999403953552
},
{
"sequence_name": "47b37eb6f9_frame_002720",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.9609947800636292,
"max_reprojection_error_px": 9.351845073979348e-05,
"max_ray_reprojection_error_px": 8.897326188161969e-05,
"centroid_distance": 1.0
},
{
"sequence_name": "4c5c60fa76_frame_006690",
"views": 8,
"resolution": 224,
"valid_depth_fraction": 0.7516716122627258,
"max_reprojection_error_px": 0.00010746094631031156,
"max_ray_reprojection_error_px": 9.423537994734943e-05,
"centroid_distance": 0.9999999403953552
}
],
"batch_shapes": {
"rgb": [
2,
8,
3,
224,
224
],
"depth": [
2,
8,
1,
224,
224
],
"depth_valid": [
2,
8,
1,
224,
224
],
"K": [
2,
8,
3,
3
],
"frame_numbers": [
2,
8
],
"T_world_from_camera": [
2,
8,
4,
4
],
"original_world_from_reference": [
2,
4,
4
],
"scene_scale": [
2
],
"points_world": [
2,
8,
3,
224,
224
],
"ray_origins": [
2,
8,
3,
224,
224
],
"ray_directions": [
2,
8,
3,
224,
224
],
"plucker": [
2,
8,
6,
224,
224
]
}
}
In canonicalization, we put every scene in a consistent coordinate system, selected as camera 0. Its pose becomes
the reference pose for all other camera poses (i.e. it becomes the identity). We then express all other camera poses
and depths relative to that reference and scale the scene accordingly.
World-space ray directions are the directions in the shared 3D world space that pixels look towards. A ray is created
from the pixel into the 3D scene.
The Plücker moment is a representation of the ray's direction and position relative to the origin, formed by m = o x d.
Q2: Novel-View Synthesis with Mini-LVSM
Overfitting
Train Loss: 0.00513
Train PSNR: 29.14
Validation Loss: 0.00021
Validation PSNR: 36.83

Training and Evaluation

| Source views | Target-ray conditioning | PSNR | SSIM | N |
|---|---|---|---|---|
| 1 | Correct | 13.97 | 0.546 | 196 |
| 1 | Shuffled | 12.11 | 0.516 | 196 |
| 1 | None (zero rays) | 12.20 | 0.544 | 196 |
| 2 | Correct | 14.29 | 0.544 | 168 |
| 2 | Shuffled | 12.05 | 0.510 | 168 |
| 2 | None (zero rays) | 12.14 | 0.540 | 168 |
| 4 | Correct | 14.44 | 0.554 | 112 |
| 4 | Shuffled | 12.19 | 0.517 | 112 |
| 4 | None (zero rays) | 12.14 | 0.550 | 112 |











Settings:
I used the default settings with some adjustments due to hardware limitations. The program ran for 100 epochs with 2 source views
and 8 total views per sample at 224x224 resolution. I increased the patch size from 8 to 16 to reduce the token size and decreased
the batch size to 1.
Increasing the number of source views generally improved image quality. As the number of source views rose in the ablations, the
PSNR also rose from 13.97 dB with one source to 14.44 dB with four sources. SSIM was also marginally higher with four sources.
Correct target rays also gave higher PSNR than either zero or shuffled target rays, which implies that incorrect target-camera
information will harm the prediction similarity. While the final results came out blocky and lacking detail, this run was only for
100 epochs due to time and hardware limitations; as the PSNR and SSIM values rose over epochs according to the training curves and
final metrics, more epochs likely would have yielded a sharper image.
Analysis and Exploration
I wanted to experiment with the number of source views as my computer hardware would allow. I tried lowering the number of source views from 2 to 1 to see how it would affect the quality of results. As expected, having less source views to train with caused a worse prediction:









| Source views | Target-ray conditioning | 1-View PSNR | 1-View SSIM | 2-View PSNR | 2-View SSIM | Δ PSNR | Δ SSIM |
|---|---|---|---|---|---|---|---|
| 1 | Correct | 13.985 | 0.5453 | 13.969 | 0.5459 | −0.015 | +0.0007 |
| 1 | Shuffled | 12.024 | 0.5144 | 12.112 | 0.5161 | +0.088 | +0.0016 |
| 1 | None | 12.342 | 0.5413 | 12.203 | 0.5437 | −0.138 | +0.0025 |
| 2 | Correct | 13.939 | 0.5431 | 14.288 | 0.5445 | +0.348 | +0.0014 |
| 2 | Shuffled | 12.083 | 0.5137 | 12.050 | 0.5104 | −0.033 | −0.0033 |
| 2 | None | 12.355 | 0.5382 | 12.136 | 0.5398 | −0.219 | +0.0016 |
| 4 | Correct | 13.659 | 0.5381 | 14.436 | 0.5536 | +0.777 | +0.0154 |
| 4 | Shuffled | 12.444 | 0.5174 | 12.186 | 0.5171 | −0.258 | −0.0003 |
| 4 | None | 12.320 | 0.5365 | 12.143 | 0.5500 | −0.177 | +0.0135 |
Due to hardware limitations, I was not able to experiment with increasing the number of source views beyond what was initially trained with, but these results suggest that there is an increase in quality that comes from the number of source views as there is more information available to the model to construct a novel view with. As shown by the table, there are mostly increases in the similarity metrics when trained with more source views.
Q3: Multi-View Geometry with Mini-VGGT
Overfitting



Training and Evaluation

| Split | Depth | Rotation AUC | Translation AUC | Joint Pose AUC | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| AbsRel | δ1 | 5° | 15° | 30° | 5° | 15° | 30° | 5° | 15° | 30° | |
| Validation | 0.1550 | 0.7901 | 0.1360 | 0.3974 | 0.6009 | 0.0059 | 0.0420 | 0.1111 | 0.0018 | 0.0270 | 0.0947 |
| Test | 0.1731 | 0.7316 | 0.1286 | 0.4259 | 0.6618 | 0.0077 | 0.0359 | 0.1146 | 0.0010 | 0.0211 | 0.0988 |








Settings:
I used the default settings with some adjustments due to hardware limitations. The program ran for 232 epochs with a batch size of
2 and 1 worker. Unfortunately, any changes to these settings proved too taxing for the computer and even this run was not able to
complete the full 1000 epochs. Provided are the best results for the last working checkpoint.
The results still approach the geometry displayed by the ground-truth views for each scene, suggesting that the full number of epochs
would have achieved a significantly higher-quality prediction. Depth generalization is reasonable according to the metrics, though not
high; this may improve over more training epochs but also shows a drop between splits. Rotation performance is stronger in test but overall
still quite inaccurate, while translation accuracy is extremely weak across both splits.
Overall, depth is often in the right range but still has a somewhat high amount of error, as only about 73-79% of pixels meet the δ1
threshold. Meanwhile, rotation error is moderate but not extremely large; still, the low 5° AUC implies that very accurate rotations are
not common. Translation has a very high amount of error in all metrics. The training loss curves show a gradual decrease in loss over epochs,
which shows that more training might have an impact on the quality of the prediction.