16-822 Assignment1

Q1: Affine Rectification (30 points)

We can get the line by using the cross product of 2 points $l=p_1\times p_2$$l = p_1 \times p_2$, and two pairs of parallel lines can be denoted as $l_1$$l_1$ and $l_2$$l_2$, and $l_3$$l_3$ and $l_4$$l_4$. Then, we can get two infinite points using these lines $p_{\mathrm{\infty }1}=l_1\times l_2$$p_{\infin 1} = l_1 \times l_2$ and $p_{\mathrm{\infty }2}=l_3\times l_4$$p_{\infin 2} = l_3 \times l_4$. From the two infinite points we can get the infinite line $l_{\mathrm{\infty }}=p_{\mathrm{\infty }1}\times p_{\mathrm{\infty }2}$$l_{\infin}=p_{\infin 1} \times p_{\infin 2}$. Now we can construct the 3x3 homography matrix by setting the third row to $l_{\mathrm{\infty }}$$l_{\infin}$ and the other diagnal values to 0.

We can show the results as follows. Each example will include the evaluate angles before and after affine rectification, and a row of visualizations: {input annotated parallel lines | affine-rectified image | input test lines | affined-rectified test lines},

Example1: consines of test lines before affine-rectification: 0.9868 and 0.9987; after affine-rectification: 0.9999 and 0.9999.

Example2: consines of test lines before affine-rectification: 0.7842 and 0.9999; after affine-rectification: 0.9999 and 0.9999.

Example3: consines of test lines before affine-rectification: 0.8964 and 0.9574; after affine-rectification: 0.9999 and 0.9999.

Example4: consines of test lines before affine-rectification: 0.9997 and 0.9911; after affine-rectification: 0.9999 and 0.9999.

Example5: consines of test lines before affine-rectification: and ; after affine-rectification: and .

Q2: Metric Rectification (40 points)

Using the affine rectification matrix from Q1, we can get the affine rectified images. And from it, we can get the metric rectification simply by using two pairs of perpendicular lines. Suppose for these two pairs of lines, each line pair can be annotated as $l=[x_1,y_1,1{]}^T$$l=[x_1, y_1, 1]^T$ and $m=[x_2,y_2,1{]}^T$$m=[x_2,y_2,1]^T$, we want to solve $\begin{array}{c}C=\left[\begin{array}{ccc}a& b/2& 0\\ b/2& c& 0\\ 0& 0& 0\end{array}\right]\end{array}$$C = \begin{bmatrix}a & b/2 & 0 \\ b/2 & c & 0 \\ 0 & 0 & 0\end{bmatrix}$ which satisfies $l^{T}Cm=0$$l^T C m =0$. Thus, for each pair we can get a condition that is $x_1{x}_{2}a+\frac{x_1{y}_2+y_1{x}_2}{2}b+y_1{y}_{2}c=0$$x_1 x_2 a + \frac{x_1y_2+y_1x_2}{2}b+y_1 y_2 c = 0$, and with two such conditions, to solve for $[a,b,c{]}^T$$[a,b,c]^T$, which is orthogonal to two conditions of $[x_1{x}_2,\frac{x_1{y}_2+y_1{x}_2}{2},y_1{y}_2]$$[x_1 x_2 , \frac{x_1y_2+y_1x_2}{2} , y_1 y_2]$, we can take the cross products of these two conditions and solve for $C$$C$. Then with $C$$C$, we can conduct singular value decomposition on it getting $U,S,V^T$$U, S, V^T$. With the two non-zero values from $S$$S$, which can be annotated as ${\sigma }_1$$\sigma_1$ and ${\sigma }_2$$\sigma_2$, we can get the metric rectification homography matrix $H_s$$H_s$ which can be calculated by $\left[\begin{array}{ccc}1/\sqrt{{\sigma }_1}& 0& 0\\ 0& 1/\sqrt{{\sigma }_2}& 0\\ 0& 0& 1\end{array}\right]$$\begin{bmatrix} 1/\sqrt{\sigma_1} & 0 & 0 \\ 0 & 1/\sqrt{\sigma_2} & 0 \\ 0 & 0 & 1\end{bmatrix}$ and $U^T$$U^T$.

We can show the results as follows. Each example will include the evaluate angles before and after metric rectification, and a row of visualizations: {input annotated perpendicular lines | affine-rectified image | metric-rectified image | input test lines | affined-rectified test lines | metric-rectified test lines},

Example1: consines of test lines before metric-rectification: -0.1677 and -0.0349; after metric-rectification: 0.0081 and 0.0293.

Example2: consines of test lines before metric-rectification: 0.1919 and -0.1065; after metric-rectification: -0.0461 and 0.0107.

Example3: consines of test lines before metric-rectification: 0.2527 and 0.0882; after metric-rectification: -0.0018 and 0.0082.

Example4: consines of test lines before metric-rectification: -0.3019 and -0.1722; after metric-rectification: -0.0347 and -0.0404.

Example5: consines of test lines before metric-rectification: 0.9184 and 0.9073; after metric-rectification: 0.0876 and 0.0213.

Q3: Planar Homography from Point Correspondences (30 points)

We want to find the $H$$H$ that can transform each point from $[x_1,y_1,1]$$[x_1, y_1, 1]$ to $[x_1^{\prime },y_1^{\prime },1]$$[x_1', y_1', 1]$. So this 3x3 matrix, whose parameters can be annotated from $h_0$$h_0$ to $h_9$$h_9$, can be solved by 4 point correspondences, where each correspondance can be expressed in matrix $\begin{array}{c}A=\left[\begin{array}{ccccccccc}-x_1& -y_1& -1& 0& 0& 0& x_1{x}_1^{\prime }& y_1{x}_1^{\prime }& x_1^{\prime }\\ 0& 0& 0& -x_1& -y_1& -1& x_1{y}_1^{\prime }& y_1{y}_1^{\prime }& y_1^{\prime }\end{array}\right]\end{array}$$A = \begin{bmatrix}-x_1 & -y_1 & -1 & 0 & 0 & 0 & x_1 x_1' & y_1 x_1' & x_1' \\ 0 & 0 & 0 & -x_1 & -y_1 & -1 & x_1 y_1' & y_1 y_1' & y_1'\end{bmatrix}$. To solve for $H$$H$, we can also apply SVD on $A$$A$, and select the last row of $V^T$$V^T$.

We can show the results as follows. Each example will include a row of visualizations: {normal Image | perspecticve image | output image}.

Example1:

Example2:

Q4: Bonus: Metric Rectification from Perpendicular Lines (10 points)

Now we do not have the affine rectified images, but we have 5 pairs of perpendicular lines. Similar as Q2, each line pair can be annotated as $l=[x_1,y_1,1{]}^T$$l=[x_1, y_1, 1]^T$ and $m=[x_2,y_2,1{]}^T$$m=[x_2,y_2,1]^T$, but we want to solve $\begin{array}{c}C=\left[\begin{array}{ccc}a& b/2& d/2\\ b/2& c& e/2\\ d/2& e/2& f\end{array}\right]\end{array}$$C = \begin{bmatrix}a & b/2 & d/2 \\ b/2 & c & e/2 \\ d/2 & e/2 & f\end{bmatrix}$ which satisfies $l^{T}Cm=0$$l^T C m =0$. Thus, for each pair we can get a condition that is $x_1{x}_{2}a+\frac{x_1{y}_2+y_1{x}_2}{2}b+y_1{y}_{2}c+\frac{x_1+x_2}{2}d+\frac{y_1+y_2}{2}e+f=0$$x_1 x_2 a + \frac{x_1y_2+y_1x_2}{2}b+y_1 y_2 c + \frac{x_1+x_2}{2}d + \frac{y_1 + y_2}{2}e + f = 0$. With five such conditions, to solve for $[a,b,c,d,e,f{]}^T$$[a,b,c,d,e,f]^T$, we can perform SVD on the 5x6 condition matrix, and set the result as the last row of $V^T$$V^T$. After getting the $a,b,c,d,e,f$$a,b,c,d,e,f$, we get the $C$$C$, and the remaining operation is same as Q2.

We can show the results as follows. Each example will include the 3 evaluate angles before and after metric rectification, and a row of visualizations: {input annotated perpendicular lines | metric-rectified image}

Example1: consines of test lines before metric-rectification: -0.1476, -0.1761 and 0.1482; after metric-rectification: 0.0063, 0.0155, and 0.0515.

Example2: consines of test lines before metric-rectification: -0.1579, -0.2694 and -0.3019; after metric-rectification: -0.0129, -0.0118, and -0.0221.

Q5: Bonus: More Planar Homography from Point Correspondences (10 points)

Using the same approach from Q3, we can overlay 3 normal images on top of an image with perspective effect by applying Q3 three times. The result is as follows:

input images:

Output image: