HW1: Projective Geometry and Homography

Q1: Affine Rectification

1. Book 1

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

2. Catheral of Learning

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

3. Chess

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

4. PNC Park (Baseball Park)

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

5. CMU Campus

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

6. Checker

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

7. Tile_3

Input Image	Annotated parallel lines on input image	Affine-Rectified Image

Evaluate Angles

Test lines on Input Image	Test lines on Affine-Rectified Image

Brief description of the implementation

The affine rectification matrix is recovered from two pairs of parallel lines. The algorithm is as follows:

1. Identify two sets of originally parallel lines in the image through annotation.
2. For each set of parallel lines, recover the equations of the lines by computing the cross product of the corresponding points.
3. For each pair of recovered lines, calculate their intersection point.
4. Identify the intersection points for each pair of lines.
5. Determine the line that passes through these intersection points, which corresponds to the imaged line at infinity.
6. To find a transformation where \(l_{\infty} = \mathbf{H}^{-T}l_{\infty}{}'\), let \(l_{\infty} = [0, 0, 1]^T\) and \(l_{\infty}{}' = [l_1, l_2, l_3]^T\), the homography matrix is given by \(\mathbf{H} = \begin{bmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ l_1 & l_2 & l_3 \end{bmatrix}\).

Q2: Metric Rectification

1. Apartment (Highland Ave.)

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

2. book1

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

3. cathedral of learning

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

4. Tile_5

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

5. Tile_3

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

6. CMU Campus

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

Brief description of the implementation

The metric rectification matrix is recovered using two pairs of parallel lines and two pairs of perpendicular lines. The process involves first performing affine rectification based on q1.py, followed by metric rectification using affine-rectified lines. The algorithm proceeds as follows:

1. Identify Orthogonal Lines: Annotate the image to identify two sets of originally orthogonal lines.
2. Recover Line Equations: For each set of orthogonal lines, recover their equations by computing the cross product of the corresponding points. Denote the recovered lines as \(l'\) and \(m'\).
3. Find Conic Dual to Circular Points: For each pair of lines, solve for the conic dual to the circular points using the relation: \(l'^T C^*_{\infty}{}' m' = 0\)
4. Affine Rectified Conic: Since the image is already affine-rectified, the conic can be expressed as: \(C^*_{\infty} = \begin{bmatrix} a & \frac{b}{2} & 0 \\ \frac{b}{2} & c & 0 \\ 0 & 0 & 0 \end{bmatrix}\). The equation becomes \(A c = 0\), where \(c = [a, b, c]^T\). With two constraints from the two orthogonal pairs, the conic can be determined by Singular Value Decomposition. The solution is the null space of \(A\).
5. Metric Rectification: Compute the SVD of the conic. Then, find the homography. \(C^*_{\infty} = H C^*_{\infty}{}' H^T\). The solution is: \(H = \begin{bmatrix} \sqrt{\sigma_1^{-1}} & 0 & 0 \\ 0 & \sqrt{\sigma_2^{-1}} & 0 \\ 0 & 0 & 1 \end{bmatrix} U^T\) Here, \(\sigma_1\) and \(sigma_2\) are the singular values from the SVD, and \(U\) is the orthogonal matrix obtained from the decomposition.

Q3: Planar Homography from Point Correspondences

Desk-Perspective

Wedding Album

Brief description of the implementation

1. Recovering Homography: The homography matrix \(H\) is recovered from four sets of correspondences. Each correspondence generates two constraints of the form \(H\mathbf{x} \times \mathbf{x}' = 0\). These constraints can be written as: $ [0  -w’^T  y’^T] = 0$ and \([w'\mathbf{x}^T \ 0 \ -x'\mathbf{x}^T] \mathbf{h} = 0\) wher \(\mathbf{h}\) is a \(9 \times 1\) vectorized representation of the homography matrix.

2. Solving for Homography: Given more than four correspondences (resulting in more than 8 constraints), the homography is solved using Singular Value Decomposition (SVD). The solution corresponds to the null space of the system of equations.

3. Warping and Pasting: Once the homography is computed, the source image is warped and pasted onto the target plane based on the homography matrix. This process is implemented in the q3.py/warp_and_paste function.

Q4: Bonus: Metric Rectification from Perpendicular Lines

Concrete Floor Blocks

Input Image	Annotated perpendicular lines	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

TownHouse

Input Image	Annotated perpendicular lines	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

Facade

Input Image	Annotated perpendicular lines	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

Tile_3

Input Image	Annotated perpendicular lines	Rectified Image

Test lines on Input Image	Test lines on Metric-Rectified Image

Brief description of the implementation

The metric rectification matrix is recovered using more than five pairs of perpendicular lines. The process is basically identical to the metric rectification of q2.py, except that it only has perpendicular line pairs.

1. Identify Orthogonal Lines: Annotate the image to identify five sets of originally orthogonal lines.
2. Recover Line Equations: For each set of orthogonal lines, recover their equations by computing the cross product of the corresponding points. Denote the recovered lines as \(l'\) and \(m'\).
3. Find Conic Dual to Circular Points: For each pair of lines, solve for the conic dual to the circular points using the relation: \(l'^T C^*_{\infty}{}' m' = 0\)
4. Affine Rectified Conic: The conic can be expressed as: \(C^*_{\infty} = \begin{bmatrix} a & \frac{b}{2} & \frac{d}{2} \\ \frac{b}{2} & c & \frac{e}{2} \\ \frac{d}{2} & \frac{e}{2} & f \end{bmatrix}\). The equation becomes \(A c = 0\), where \(c = [a, b, c, d, e, f]^T\). The conic can be determined by Singular Value Decomposition. The solution is the null space of \(A\).
5. Metric Rectification: Compute the SVD of the conic. Then, find the homography. \(C^*_{\infty} = H C^*_{\infty}{}' H^T\). The solution is: \(H = \begin{bmatrix} \sqrt{\sigma_1^{-1}} & 0 & 0 \\ 0 & \sqrt{\sigma_2^{-1}} & 0 \\ 0 & 0 & 1 \end{bmatrix} U^T\) Here, \(\sigma_1\) and \(sigma_2\) are the singular values from the SVD, and \(U\) is the orthogonal matrix obtained from the decomposition.

Q5: Bonus: More Planar Homography from Point Correspondences

My wedding photos posted in a movie theater.

Original Image

Generated Image