Assignment 2 - prabhdes - 16-822: Geometry-based Methods in Vision

Q1: Camera matrix P from 2D-3D correspondences

Q1 (a) Stanford Bunny

The projection matrix computed from the 2D to 3D correspondences is: [[ 6.4316938e+03 -2.9484377e+03 1.1465812e+03 2.2272434e+03]
[-9.3481921e+02 -6.7548647e+03 2.0294906e+03 1.8221877e+03]
[ 5.7930738e-01 -1.4238538e+00 -7.3526835e-01 1.0000000e+00]]

Annotated Image	Annotated Points	Surface Points	Bounding Box

Q1 (b) Cuboid

The projection matrix computed from the 2D to 3D correspondences is:
[[ 1.08025764e+02 5.87190866e+00 -5.46413708e+00 5.36429558e+01]
[-4.56150780e+01 1.26173729e+02 4.30964775e+01 -1.62894211e+01]
[ 6.65985346e-02 2.69514415e-02 -6.11318350e-02 1.00000000e+00]]

Annotated Image	Annotated Points	Bounding Box

Q2 Camera calibration K from annotations

Q2 (a) Camera calibration from vanishing points

Input Image	Annotated Image	Bounding Box

Calibration Matrix (K):
[[1.154178e+03 0.000000e+00 5.750660e+02]
[0.000000e+00 1.154178e+03 4.319391e+02]
[0.000000e+00 0.000000e+00 1.000000e+00]]
Principal Point: [575.066, 431.9391]

Algorithm:

1. For each pair of parallel lines in the scene, we compute the vanishing point v by taking their intersection:
   $v = l \times m = 0$
2. We assume that the camera intrinsic matrix K is a square matrix with zero skew and square pixels.
3. Consequently, the image of the absolute conic w, which is given by $w = K^{-T} K^{-1}$, will be of the form:
   [[a, 0, b]]
   [0, a, c]
   [b, c, d]
4. From each pair of orthogonal vanishing points, we derive constraints on w. Specifically, for each pair of vanishing points $v_i$ and $v_j$, we have the following constraint: $v_i^Twv_j=0$​
5. We construct a system of equations Ac=0, where c=[a,b,c,d]T represents the unknowns. Using SVD, we solve for c and thus obtain the elements of w.
6. Finally, we recover the camera intrinsic matrix K by performing Cholesky decomposition on w, as: $w = K^{-T} K^{-1}$

Q2 (b) Camera calibration from metric planes

Input Image	Annotated Square 1	Annotated Square 2	Annotated Square 3

Calibration Matrix (K):
[[ 1.0802634e+03 -7.6256843e+00 5.1338269e+02]
[ 0.0000000e+00 1.0769381e+03 3.9484299e+02]
[ 0.0000000e+00 0.0000000e+00 1.0000000e+00]]

Algorithm:

1. For each square, we assume rectified coordinates as (0,0),(1,0),(1,1),(0,1), and compute the homography matrix H that maps these points to the annotated points in the image. For the three annotated squares, we compute three homographies H1, H2, H3.
2. The homographies provide constraints on the image of the absolute conic w, which is given by $w = K^{-T} K^{-1}$. For each homography $H=[h_1, h_2, h_3]$, we derive two constraints as follows:
   $h_1^Twh_2 = 0$
   $h_1^Twh_1 = h_2^Twh_2$
3. The constraints derived from the homographies can be written in the form Ac=0, where c is a flattened vector of the unknowns in w. We solve this system using SVD to obtain w.
4. Once w is computed, we use Cholesky decomposition to recover the camera intrinsic matrix K as: $w = K^{-T} K^{-1}$

Q3 Single View Reconstruction

K =
[[808.52 0. 510.71]
[ 0. 808.52 363.63]
[ 0. 0. 1. ]]

Input Image	Annotated Image	Reconstructed View 1	Reconstructed View 2	Reconstructed View 3

Algorithm:

1. Using the method described in Q2a, we first compute the camera's intrinsic matrix K from three pairs of perpendicular vanishing lines corresponding to the planes.
2. For each plane, compute the normal vector n by using the relationship $n = K^Tv$, where v is the vanishing line of the plane.
3. For all points on the plane, compute the rays extending from the camera center through the image points. These rays are obtained using: $r = K^{-1}[x, y, 1]^T$
4. Using the known 3D reference point (1) and the normal vector, we compute the plane equation in the form $n^T + x = 0$.
5. Find the intersection of the computed rays with the plane to determine the 3D coordinates of all points on the plane.
6. For each additional plane, select a point that is common with another plane, and repeat steps 5-7. This ensures the consistency of the 3D reconstruction across all planes.
7. Iterate through all annotated planes, repeating the steps above to compute the 3D coordinates of all points.