Assignment 2

Andrew ID: roshanr

Late Days: 2

1. Camera matrix P from 2D-3D correspondences

Bunny

Recovered P

[ 4.06890104e+03 -1.82737181e+03  7.89101392e+02  1.42362371e+03]
[-6.46096808e+02 -4.29789393e+03  1.36401935e+03  1.16583711e+03]
[ 3.41890021e-01 -8.80611490e-01 -4.47623375e-01  6.39128040e-01]

Cuboid

Sampled 3D points projected onto the image!

Recovered P

[ 11.07030137   1.50335467  -0.37252062  74.00494539]
 [  0.19740798   0.08010682  -6.95123336 141.13111233]
 [  0.00058421   0.00676127  -0.00232587   1.        ]

2. Camera Calibration K from Annotations

PART (a)

Recovered K

[ 1.15417802e+03  0.00000000e+00  5.75066005e+02]
 [-0.00000000e+00  1.15417802e+03  4.31939090e+02]
 [ 4.45651455e-16  0.00000000e+00  1.00000000e+00]

Implementation

1. Vanishing Points Calculation
   • For each pair of points, calculate the parallel lines, as $l = x_1 \times x_2$. The intersection of the parallel lines results in a vanishing point $v = l_1 \times l_2$. Do the same for the other two vanishing points to get $v_1, v_2, v_3$.

2. Image of absolute conic ($\omega$) Calculation

   $\omega$ is of the form:

   $$\omega = \begin{bmatrix} a & b & c \\ b & d & e \\ c & e & f \end{bmatrix}$$

   Since pixels are square and have 0 skew, $a = d$ and $b=0$$b = 0$$\omega$ has 3 degrees of freedom (-1 for scale ambiguity). We must obtain 3 constraints to recover $\omega$.

   The three vanishing points are pair-wise orthogonal to each other. All pairs $(v_i, v_j)$ satisfy $v_i^{T}\omega v_j = 0$

   $$\begin{bmatrix} v_{i0}v_{j0} + v_{i1}v_{j1} & v_{i0}v_{j2} + v_{i2}v_{j0} & v_{i1}v_{j2} + v_{i2}v_{j1} & v_{i2}v_{j2} \end{bmatrix} \begin{bmatrix} a \\b \\ c\\d\end{bmatrix} = 0$$

   Three pairs of orthogonal points provide three pairs of constraints, and we get a system of equations for the form:

   $$Ax=0$$

   $x$ is calculated as the right singular vector of the $v$ matrix

3. Recover K from $\omega$$\omega = K^{-T}K^{-1}$

   Perform Cholesky decomposition to get $\omega = LL^{T}$ where $K = L^{-T}$

   PART (b): Camera calibration from metric planes

Recovered K

[ 1.07692614e+03 -4.52638065e+00  5.11568923e+02]
 [ 1.19275729e-13  1.07626752e+03  3.95526278e+02]
 [ 4.67331363e-16 -1.82919671e-18  1.00000000e+00]

Angle between planes 0 and 1: 67.282425264
Angle between planes 1 and 2: 92.203485588
Angle between planes 2 and 3: 94.710693016

Implementation

1. We are given multiple metric planes. We must calculate the homography that takes the square defined at $[[0,0], [0,1], [1,1], [1,0]]$to the annotated points on the image square.

2. For a given square with calculated homography $H$

   $$h_1^T \omega h_2 = 0; h_1^T \omega h_1 = h_2^T \omega h_2$$

Set $\omega = \begin{bmatrix} a & b & c \\ b & d & e \\ c & e & f \end{bmatrix}$and calculate based on the constraints.

Let $h_1^T = [x_1, x_2, x_3]$ and $h_2^T = [y_1, y_2, y_3]$ then we get the equations

We need three point pairs to provide 6 constraints, and we get a system of equations of the form:

3. Recover K from $\omega$ the same way as the previous question

4. Angle between planes calculation:
   • Find the normal vector that characterizes both planes.

     Given vanishing points are $(v_1, v_2)$ of two sets of parallel lines:

     $$v_l = v_1 \times v_2 \\ v_l = \omega v_3 \\ v_3 = \omega^{-1} v_l \\ n = K^{-1}v_3$$

   Angle between 2 planes is the angle between their normals

   $$cos(\theta) = \frac{n_1^Tn_2}{\sqrt{n_1^Tn_1*n_2^Tn_2}}$$

PART (c) Camera calibration from rectangles with known sizes

3. Single View Reconstruction

Recovered K

[809.95140903   0.         510.72008772]
 [ -0.         809.95140903 356.2689548 ]
 [ -0.           0.           1.        ]

Implementation

• Calculate the K matrix using Q2a. Use 3 pairs of parallel lines from planes 1 and 2.

• Calculate the normal direction of each plane from Q2b.

• Compute the intercept of each plane by choosing a corner, calculating the direction of the ray passing through the corner, and setting the 3d depth as 1 unit along that direction.

  $$d = K^{-1}x$$

• Using the plane normal direction, plane intercept, and known 3d depth of the corner point $X_w$, calculate the plane equation $\pi$

  $$n^{T} = b$$

• Compute the 3D coordinate of all points on the plane, via ray-plane intersection of each ray with the plane equation.

  $$X = \frac{b}{n^{T}ray}(d)$$

• Repeat the process for other planes. We do not need to ground a new point on each plane as there are some common points between the planes.