Assignment 2

Andrew ID: bjhamb

Q1: Camera matrix P from 2D-3D correspondences (30 points)

Bunny:

Calculated Matrix P:


	 [ 6363.7512755  -2857.91004589  1238.41308267  2227.47152199]
	 [-1013.06892585 -6724.20242552  2137.74831366  1824.17003515]
	 [    0.53358345    -1.37704085    -0.69822705     1.        ]

Cuboid

Computed P

[ 162.23534225   75.19758457  -39.84539248  708.00144083]
 [   8.78359361  -11.27977983 -173.13125277 1785.80830353]
 [   0.00423805    0.0430678    -0.02390775    1.        ]

I sampled 3d points within cuboid and projected them to the image, here is the visualization

Q2: Camera calibration K from annotations (40 points + 10 points bonus)

(a) Camera calibration from vanishing points (20 points)

Provided Annotations:

Recovered Camera Intrinsics Matrix (K):

 [1154.17801827    0.          575.06600499]
 [   0.         1154.17801827  431.93909042]
 [   0.            0.            1.        ]

Custom Annotations

Recovered K:

 [1002.45084201    0.          733.63881232]
 [   0.         1002.45084201  528.95733584]
 [   0.            0.            1.        ]

The recovered K is a bit different from the K found above, indicating that results are sensitive to the annotations

Implementation Steps

Find Vanishing Points
We are provided 3 pairs of parallel lines, all of which are perpendicular to each other.
- For each pair, extract parallel lines from gives points, line passing through points $p_1$ and $p_2$ can be found using $l_1 = p_1 \times p2$ and $l_2 = l_1 = p_3 \times p4$ . Find Intersection point of pair of parallel lines using $v = l_1 \times l_2$ . This is the vanishing point for the direction corresponding to the parallel lines
- Repeat the above to find the 3 vanishing points $v_1, v_2, v_3$

Find the Image of absolute conic ( $\omega$ )
Since $\omega$ is a symmetric matrix, we can write it as:
$\omega = \begin{bmatrix} a & b & c \\ b & d & e \\ c & e & f \end{bmatrix}$
As we are given that the pixels are square and have 0 skew $\implies$ a=d and b=0
$\omega = \begin{bmatrix} a & 0 & c \\ 0 & a & e \\ c & e & f \end{bmatrix}$
Thus $\omega$ has 3 degrees of freedom (4-1(for scale ambiguity)) and we need 3 constraints to find it.

Since all three vanishing are orthogonal to each other, for any pair $(v_i, v_j)$ we can say
$v_i^{T}\omega v_j = 0$
$\begin{bmatrix} v_{i0}v_{j0} + v_{i1}v_{j1} & v_{i0}v_{j2} + v_{i2}v_{j0} & v_{i1}v_{j2} + v_{i2}v_{j1} & v_{i2}v_{j2} \end{bmatrix} \begin{bmatrix} a \\b \\ c\\d\end{bmatrix} = 0$

Thus each pair adds 1 constraint on the iac, and three pairs provide the 3 constraints needed to find $\omega$ . We can stack each constraints to get an equation of the form
$Ac=0$
c (and hence can be found as the right singular vector of A

Find K from $\omega$
Since $\omega = K^{-T}K^{-1}$
We can do cholesky decomposition to get $\omega = LL^{T}$
Thus K would be $K = L^{-T}$
Principle point would just be (K[0,2], K[1,2])

b) Camera calibration from metric planes (20 points)

Recovered K:

 [1076.92614406   -4.52638065  511.56892341]
 [   0.         1076.26752438  395.52627813]
 [   0.            0.            1.        ]

Angle between planes 0 and 1: 67.2824252644341
Angle between planes 1 and 2: 92.20348558861174
Angle between planes 2 and 3: 94.71069301662284

Implementation Steps:

For each metric plane, identify homography that transforms a metric square (A square with coordinates {(0,1), (1,1), (1,0), (0,0)} to the annotated squares.

Each such homography provides 2 constraints.
- If $H = (h_1, h_2, h_3)$ where $(h_1, h_2, h_3)$ are columns of H, then :
$h_1^T \omega h_2 = 0$

h_1^T \omega h_1 = h_2^T \omega h_2

If $h_1^T = [x_1, x_2, x_3]$ and $h_2^T = [y_1, y_2, y_3]$ then the constraints can be written as

If $\omega = \begin{bmatrix} a & b & c \\ b & d & e \\ c & e & f \end{bmatrix}$

\begin{bmatrix} x_1 x_2 & x_1y_2 + y_1x_2 & x_1 w_2 + w_1 x_2 & y_1 y_2 & y_1 w_2 +w_2y_1 & w_1 w_2 \\ x_1^2-x_2^2 & 2(x_1y_1-x_2y_2) & 2(x_1w_1-x_2w_2) & y_1^2-y_2^2 & 2(y_1w_1-y_2w_2) & w_1^2-w_2^2 \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \end{bmatrix} = 0

Thus each pair adds 2 constraint on the iac, and three pairs provide the 6 constraints needed to find $\omega$ . We can stack each constraints to get an equation of the form

Ac=0

c (and hence can be found as the right singular vector of A

Find K from $\omega$
Since $\omega = K^{-T}K^{-1}$
We can do cholesky decomposition to get $\omega = LL^{T}$
Thus K would be $K = L^{-T}$

Find angles
- Find normals to each plane
  For each plane, identify 2 pairs of parallel lines, find intersection of each pair to find the vanishing points. Each vanishing point gives a direction and we can find normal to the plane by taking cross product of the 2 directions.
  Say vanishing points are $(v_1, v_2)$ then:
  $d_1 = K^{-1}v_1 \\ d_2 = K^{-1}v_2 \\ normal = d_1 \times d_2$
Then to find angle between 2 planes, we can simply find angle between their normals
$cos(\theta) = \frac{n_1^Tn_2}{\sqrt{n_1^Tn_1*n_2^Tn_2}}$

c) Camera calibration from rectangles with known sizes (10 points bonus)

Recoverd K

 [3142.59911967 -438.70165602 2876.60285282]
 [   0.         3073.06738741 2157.30593526]
 [   0.            0.            1.        ]

Angle between planes 0 and 1: 63.89607900586695
Angle between planes 1 and 2: 24.173714486301964
Angle between planes 2 and 3: 39.9442024479043

Implementation Steps:

The steps are same as the previous question. The only difference is the vertices plane from which homography is calculated. Instead of being a metric plane now it is a rectangle with coordinates

width = 1
height = width * height*height_width_ratio
Vertices: [[0, height], [width, height], [width, 0], [0, 0]]

Q3: Single View Reconstruction (30 points + 10 points bonus)

Implementation Steps:

Use Q2a to compute K (find 3 pairs of parallel lines that are orthogonal to each other and then use Q2a)

Find normal for each plane as described in Q2b

Compute ray for each point
- Equation of a ray given a pixel
  $ray = K^{-1}x$
- Pick one point as reference and set its depth

Compute plane equation given the known 3D point.
- Equation of plane
$n^{T} = d$

Compute 3D coordinate of all points on the plane via ray-plane intersection.
- Given ray r and a plane with normal and intercept (n,d), the intersectioj point can be found as
$point = \frac{d}{n^{T}ray}(ray)$

Use the above to recreate each point on plane as well as other planes

Recovered K:

 [809.95140903   0.         510.72008772]
 [  0.         809.95140903 356.2689548 ]
 [  0.           0.           1.        ]

(b) (10 points bonus)