HW1: Projective Geometry and Homography

Q1: Affine Rectification

Description

Step 1: Normalize the pixel coordinates.

While not strictly required for affine rectification, normalizing pixel coordinates can help produce transformations with smaller coefficients, reducing image distortion.

To do this, compute a similarity transformation \(T\) such that the centroid of the points is moved to the origin, and the maximum distance from the origin is scaled to \(\sqrt{2}\).

Step 2: Find \(l_\infty\)

To compute the line at infinity \(l_\infty\), identify two points that lie on \(l_\infty\) using the given pairs of parallel lines. Specifically, compute the intersections \(l_1 \times l_2\) and \(l_3 \times l_4\), where \((l_1, l_2)\) and \((l_3, l_4)\) are the two pairs of parallel lines. The line \(l_\infty\) is then determined by these points:

\[l_\infty = (l_1 \times l_2) \times (l_3 \times l_4)\]

Step 3: Compute \(\mathrm{H}\)

The matrix \(\mathrm{H}\) for normalized coordinates can be computed as:

\[ \mathrm{H} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ l_1 & l_2 & l_3 \end{bmatrix} \]

The matrix \(\mathrm{H}\) for the original coordinates is then given by \(\mathrm{T}^{-1} \mathrm{H} \mathrm{T}\).

Output Images

Instance	Input Image	Annotated parallel lines on input image	Affine-Rectified Image
Book
Checker
Facade
Ground
Cabinet

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

Q2: Metric Rectification

Description

Step 1: Apply affine rectification and normalize the pixel coordinates.

Apply the affine rectification from Q1 to the points labeled.

Similar to Q1, we compute a similarity transformation \(T\) such that the centroid of the points is moved to the origin, and the maximum distance from the origin is scaled to \(\sqrt{2}\).

Step 2: Compute \(C_\infty^{*'}\)

After the affine rectification, \(C_\infty^{*'}\) has the form:

\[ C_\infty^{*'} = \begin{bmatrix} a & b & 0 \\ b & c & 0 \\ 0 & 0 & 0 \end{bmatrix} \]

For two orthogonal lines \(\mathbf{l}\) and \(\mathbf{m}\), we have \(\mathbf{l}^T C_\infty^{*'} \mathbf{m} = 0\).

Expanding it out, we get:

\[ a l_1 m_1 + c l_2 m_2 + b(l_1 m_2 + l_2 m_1) = 0 \]

Therefore, for two pairs of perpendicular lines \((\mathbf{l}^1, \mathbf{m}^1)\) and \((\mathbf{l}^2, \mathbf{m}^2)\), we can compute:

\[ (a, b, c) = (l_1^1 m_1^1, l_1^1 m_2^1 + l_2^1 m_1^1, l_2^1 m_2^1) \times (l_1^2 m_1^2, l_1^2 m_2^2 + l_2^2 m_1^2, l_2^2 m_2^2) \]

Step 3: Compute \(\mathrm{H}\)

After deriving \(C_\infty^{*'}\), we can compute \(\mathrm{H}\) using SVD. Specifically, let:

\[ C_\infty^{*'} = U \begin{bmatrix} \sigma_1 & 0 & 0 \\ 0 & \sigma_2 & 0 \\ 0 & 0 & 0 \end{bmatrix} U^\mathsf{T} \]

\(\mathrm{H}\) can be computed as:

\[ \mathrm{H} = \begin{bmatrix} \sqrt{\sigma_1^{-1}} & 0 & 0 \\ 0 & \sqrt{\sigma_2^{-1}} & 0 \\ 0 & 0 & 1 \end{bmatrix} U^\mathsf{T} \]

Output Images

Instance	Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image
Book
Checker
Facade
Ground
Cabinet

Evaluate Angles

Test lines on Input Image	Test lines on Metric-Rectified Image

Q3: Planar Homography from Point Correspondences

Description

Step 1: Normalize the pixel coordinates.

Apply the affine rectification from Q1 to the points labeled, although this step is not required to estimate homography from 4 point correspondences.

Similar to Q1, we compute a similarity transformation \(T\) such that the centroid of the points is moved to the origin, and the maximum distance from the origin is scaled to \(\sqrt{2}\).

Step 2: Formulate equations for \(\mathrm{H}\)

For each pair of correspondence \(((x_1,y_1),(x_2,y_2))\), we have \(\mathrm{H}{(x_1,y_1)} \times (x_2,y_2) = 0\)

Expanding it out, we have \(x_1H_{11}+y_1H_{12}+H_{13}-x_1x_2H_{31}-x_2y_1H_{32}-x_2H_{33} = 0\) and \(x_1H_{21}+y_1H_{22}+H_{23}-x_1y_2H_{31}-y_1y_2H_{32}-y_2H_{33} = 0\).

We can formulate these equations as \(\mathrm{A}\mathrm{H} = 0\)

Step 3: Solve \(\mathrm{H}\)

We simply use SVD to solve \(\mathrm{H}\), the homography for unnormalized coordinates is given by \(\mathrm{T_2}^{-1}\mathrm{H}\mathrm{T_1}\)

Output Images

Normal Image	Perspective Image	Annotated corners in Perspective image	Warped and Overlaid Image

Q4: Bonus: Metric Rectification from Perpendicular Lines

Description

Step 1: Normalize the pixel coordinates.

This step is required for this problem.

Similar to Q1, we compute a similarity transformation \(T\) such that the centroid of the points is moved to the origin, and the maximum distance from the origin is scaled to \(\sqrt{2}\).

Step 2: Compute \(C_\infty^{*'}\)

Let \[C_\infty^{*'}=\left[\begin{array}{ccc}a&b&d\\b&c&e\\d&e&f\end{array}\right]\]

For each pair of orthogonal lines \((\mathrm{l},\mathrm{m})\), we have \(\mathrm{l}^TC_\infty^{*'}\mathrm{m} = 0\), which is a linear equation with respect to the elements in \(C_\infty^{*'}\)

Combining all the equations together, we get a linear equation \(\mathrm{A}[a,b,c,d,e,f]^T = 0\)

We can find the \(C_\infty^{*'}\) which minimizes \(\mathrm{A}[a,b,c,d,e,f]^T\) when \(||a,b,c,d,e,f||_2=1\) using the SVD decomposition

Step 3: Solve \(\mathrm{H}\)

As shown in Q2, we can derive \(\mathrm{H}\) from \(C_\infty^{*'}\) from SVD decomposition

Output

Input Image	Annotated perpendicular lines	Rectified Image

Evaluate Angles

Q5: Bonus: More Planar Homography from Point Correspondences

Description

Run the code in Q3 multiple times to overlay multiple images.

Output

Normal Image 1	Normal Image 2	Normal Image 3	Perspective Image	Annotated points	Warped and Overlaid Image