Assignment 1

Roshan Roy (roshanr)

Late Days: 1

Q1. Affine Rectification

Implementation Explained

1. Calculate lines from consecutive points

   $l = p_1 \times p_2$

2. For lines $l_1$ and $l_2$, calculate the intersection point $x_1$. For lines $l_3$ and $l_4$, calculate the intersection point $x_2$.

   $x_1 = l_1 \times l_2$

3. Calculate the image of the line at infinity

   $l_{\infin} = x_1 \times x_2$

4. The homography $H$ such that $H^{-T}L$ becomes $[0, 0, 1]^T$ is

   $$H = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ l_1 & l_2 & l_3 \\ \end{bmatrix}$$

5. Warp the entire image with the homography

Tiles 5

Tiles 3

Facade

Custom Image 1

Custom Image 2

Q2. Metric Rectification

Implementation Explained

1. Calculate lines from consecutive points. There should be 8 lines.

   $l = p_1 \times p_2$

2. Perform affine rectification of the image.

3. Find ${C_{\infty}^{*}}^{'}$ for perpendicular lines $l,m$ such that

Since affine rectification of lines has already occurred, the matrix is expressed as:

4. Collect 2 additional pairs of points for metric rectification since there are 2 remaining DoF. For each pair,

Setup the A matrix where each row contains a constraint from a pair of points.

5. Solve $Ax = 0$ using SVD to find ${C_{\infty}^{*}}^{'}$

6. Use SVD on ${C_{\infty}^{*}}^{'}$ to split it into matrices $U, \Sigma, V^T$. The rectification matrix $H$ is given by:

Tiles 5

Tiles 3

Facade

Custom 1

Custom 2

cosine before	cosine after
-0.2794581244460149	-0.03336564977793258
0.6695406065290532	0.08244873784229738

Q3. Planar Homography

Implementation Details

First, we normalize the points in both images to bring it to similar scales.

For a given pair of correspondences,

We introduce the constraint

After expanding and massaging this equation (where h is an unrolled vector of 9 elements):

Note that this only gives us two constraints, not 3.

Given 4 point correspondences (H has 8 DoF), we stack the system of linear equations into the form $Ah = 0$. We solve with the constraint $||h|| = 1$ to avoid the trivial solution. Solve for h using SVD (h is the right singular vector of V).

Book on Table

Custom Image

Q4. Metric Rectification with Perpendicular Lines

Implementation Steps

1. Calculate lines from consecutive points. Calculate all 8 lines:

   $l_1 = p_1 \times p_2$

2. Find ${C_{\infty}^{*}}^{'}$ for perpendicular lines $l,m$ such that ${l^{'}}^T {C_{\infty}^{*}}^{'} m = 0$

   ${C_{\infty}^{*}}^{'}$ is of the form

   $${C_{\infty}^{*}}^{'} = \begin{bmatrix} a & b & d \\ b & c & e \\ d & e & f \end{bmatrix}$$

3. Collect 5 pairs of perpendicular points for metric rectification since there are 5 DoF, and since each pair gives one constraint. For each pair:

   $$\begin{bmatrix} l_1m_1 & l_1m_2+l_2m_1 & l_2m_2 & l_1m_3+l_3m_1 & l_2m_3+l_3m_2 & l_3m_3 \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\e \\f \end{bmatrix} = 0$$

4. Setup the A matrix where each row contains a constraint from a pair of points.

5. Solve $Ax = 0$ using SVD to find ${C_{\infty}^{*}}^{'}$

6. Use SVD on ${C_{\infty}^{*}}^{'}$ to split it into matrices $U, \Sigma, V^T$. The rectification matrix $H$ is given by:

7. Warp the image with this $H$ matrix.

Tiles 5

Custom Image

Q5. Bonus

A picture of my old roommate’s dog Rosy on multiple Times Square billboards. Implementation is similar to that of Q3, just applied multiple times. Please refer to the code for more details.