Assignment 0

Task 1: Affine Rectification

Input image	Annotated parallel lines on input image	Affine-Rectified image

Verification

Test lines on input image	Test lines on Affine-Rectified image	Angles
		Before After Red      0.999989      0.999993 Blue      0.959984      0.995298
		Before After Red      0.784232      0.999978 Blue      0.999997      0.999986
		Before After Red      0.896445      0.999994 Blue      0.957452      0.999973
		Before After Red      0.999644      0.999970 Blue      0.940165      0.999824
		Before After Red      0.961854      0.999901 Blue      1.0      0.999222

Implementation

1. For each pair of parallel lines, compute the vanishing point.
2. Use 2 vanishing points to compute the line at infinity \[l_{\infty}' = \begin{bmatrix} l_0\\ l_1\\ l_2 \end{bmatrix}\]
3. According to \( H^T l_{\infty} = l_{\infty}'\), compute \[ H = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ l_0 & l_1 & l_2 \end{bmatrix} \]

Task 2: Metric Rectification

Input image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified image	Rectified image

Verification

Test lines on input image	Test lines on Metric-Rectified image	Angles
		Before After Red      -0.118708      0.028819 Blue      0.160231      -0.016235
		Before After Red      -0.191871      0.046072 Blue      -0.106548      0.010747
		Before After Red      -0.252671      0.001817 Blue      0.088174      0.008199
		Before After Red      -0.786999      -0.057912 Blue      0.775755      -0.004344
		Before After Red      0.276967      -0.044835 Blue      0.343193      -0.039958

Implementation

1. For each pair of perpendicular lines \(l\) and \(m\). \[ l = \begin{bmatrix} l_0\\ l_1\\ l_2 \end{bmatrix} \quad \quad \quad m = \begin{bmatrix} m_0\\ m_1\\ m_2 \end{bmatrix} \] According to the function \[ l^T C_{\infty}^*{}' m = 0\] Since the image is already affine rectified, We have \[ C_{\infty}^*{}' = \begin{bmatrix} a & b & 0 \\ b & c & 0 \\ 0 & 0 & 0 \end{bmatrix} \implies C = { \begin{bmatrix} a & b & c \end{bmatrix}}^T \] Thus, we only need 2 pairs of perpendicular lines. For each pair of \(l\) and \(m\), we have \[ A_{sub}C = 0 \quad \quad \text{where} \quad \quad A_{sub} = \begin{bmatrix} l_0 * m_0 & l_0 * m_1 + l_1 * m_0 & l_1 * m_1 \end{bmatrix} \] We have 2 pairs of \(l\) and \(m\) and concatenate 2 \(A_{sub}\) in to a big matrix \(A\).
2. Conduct SVD over \(A\) accoridng to \(A = U \Sigma Vt\), extract the last row of \(Vt\) according to \(C = Vt[-1]\).
3. Conduct SVD over \(C_{\infty}^*{}'\) accoridng to \[ C_{\infty}^*{}' = U \begin{bmatrix} \sigma_1 & 0 & 0 \\ 0 & \sigma_2 & 0 \\ 0 & 0 & 0 \end{bmatrix} Ut \]
4. Considering \(C_{\infty}^* = H C_{\infty}^*{}' H^T\), calculate \(H\) according to \[ H = \begin{bmatrix} \sqrt{\sigma_1^{-1}} & 0 & 0 \\ 0 & \sqrt{\sigma_2^{-1}} & 0 \\ 0 & 0 & 1 \end{bmatrix} Ut \]
5. Warp the input image using \(H\).

Task 3: Planar Homography from Point Correspondences

Normal image	Perspective image	Annotated corners in perspective image	Warped and overlaid image

Implementation

1. Set the four corners of the source image as \( a\), the corresponding points in the target image as \( a'\). \[ a = \begin{bmatrix} x\\ y\\ 1 \end{bmatrix} \quad \quad \quad a' = \begin{bmatrix} x'\\ y'\\ 1 \end{bmatrix} \] According to the function \[ Ha \times a' = 0\] We have \[ H = \begin{bmatrix} h_1 & h_2 & h_3 \\ h_4 & h_5 & h_6 \\ h_7 & h_8 & h_9 \end{bmatrix} \implies h = { \begin{bmatrix} h_1 & h_2 & h_3 & h_4 & h_5 & h_6 & h_7 & h_8 & h_9 \end{bmatrix} }^T \] For each pair of \(a\) and \(a'\), we have \[ A_{sub}h = 0 \quad \quad \text{where} \quad \quad A_{sub} = \begin{bmatrix} 0 & 0 & 0 & -x & -y & -1 & x*y' & y*y' & y' \\ x & y & 1 & 0 & 0 & 0 & -x*x' & -y*x' & -x' \end{bmatrix} \] We have 4 pairs of \(a\) and \(a'\) and concatenate 4 \(A_{sub}\) in to a big matrix \(A\).
2. Conduct SVD over \(A\) accoridng to \(A = U \Sigma Vt\), extract the last row of \(Vt\) according to \(h = Vt[-1]\), normalize it and reshape it to get \(H\).
3. Warp the source image to the same size of the target image using \(H\), then use mask to combine the warped image and the source image.

Task 4: Metric Rectification from Perpendicular Lines

Input image	Annotated perpendicular lines	Rectified image

Verification

Test lines on input image	Test lines on Metric-Rectified image	Angles
		Before After Red      -0.0177778      0.0373943 Blue      0.1726906      -0.0506602 Green      0.0      -0.0570613
		Before After Red      0.6957433      -0.0986491 Blue      0.8418337      0.1072315 Green      0.6977939      -0.1144775

Implementation

It differs with Task 2 in that

1. \[ C_{\infty}^*{}' = \begin{bmatrix} a & b & d \\ b & c & e \\ d & e & f \end{bmatrix} \implies C = { \begin{bmatrix} a & b & c & d & e & f \end{bmatrix}}^T \]
2. \[ A_{sub} = \begin{bmatrix} l_0 * m_0 & l_0 * m_1 + l_1 * m_0 & l_1 * m_1 & l_0 * m_2 + l_2 * m_0 & l_1 * m_2 + l_2 * m_1 & l_2 * m_2 \end{bmatrix} \]
3. We have 5 pairs of \(l\) and \(m\) and concatenate 5 \(A_{sub}\) in to a big matrix \(A\).

Task 5: More Planar Homography from Point Correspondences

Normal images
Perspective image	Annotated corners in perspective image	Warped and overlaid image

Implementation

Sequentially apply planar homography to normal images one by one.