Assignment 1¶
Yixin Fei (yixinf)¶
tiles5:
| Input Image | Annotated parallel lines on input image | Affine-Rectified Image |
|---|---|---|
 |
 |
 |
facade:
| Input Image | Annotated parallel lines on input image | Affine-Rectified Image |
|---|---|---|
 |
 |
 |
book1:
| Input Image | Annotated parallel lines on input image | Affine-Rectified Image |
|---|---|---|
 |
 |
 |
myfacade2:
| Input Image | Annotated parallel lines on input image | Affine-Rectified Image |
|---|---|---|
 |
 |
 |
mychess2:
| Input Image | Annotated parallel lines on input image | Affine-Rectified Image |
|---|---|---|
 |
 |
 |
Q1.2 Evaluating angles and visualization¶
tiles5:
| Before | After |
|---|---|
| 0.986833 | 0.999925 |
| 0.998719 | 0.999938 |
| Test lines on Input Image | Test lines on Affine-Rectified Image |
|---|---|
 |
 |
facade:
| Before | After |
|---|---|
| 0.784232 | 0.999978 |
| 0.999997 | 0.999987 |
| Test lines on Input Image | Test lines on Affine-Rectified Image |
|---|---|
 |
 |
book1:
| Before | After |
|---|---|
| 0.999989 | 0.999993 |
| 0.959984 | 0.995299 |
| Test lines on Input Image | Test lines on Affine-Rectified Image |
|---|---|
 |
 |
myfacade2:
| Before | After |
|---|---|
| 0.997871 | 0.999606 |
| 0.999219 | 0.999876 |
| Test lines on Input Image | Test lines on Affine-Rectified Image |
|---|---|
 |
 |
mychess2:
| Before | After |
|---|---|
| 0.994966 | 0.999966 |
| -0.997000 | -0.998019 |
| Test lines on Input Image | Test lines on Affine-Rectified Image |
|---|---|
 |
 |
Q1.3 Brief description of the implementation¶
Given 2 pairs of annotated parallel lines, $p_{11}$, $p_{12}$, $p_{21}$, $p_{22}$, $p_{31}$, $p_{32}$, $p_{41}$, $p_{42}$, we can first get 4 lines by computing the cross product of each of two points in homogeneous coordinates (eg. $p_{11}$, $p_{12}$). Two pairs of lines can be annotated as: $l_1$, $l_2$, $m_1$, $m_2$.
These two sets of parallel lines intersect at points $p_1$, $p_2$. We can get in homogeneous coordinates:
$$p_1 = l^1 \times l^2$$
$$p_2 = m^1 \times m^2$$
The vanishing line $l=(l_1, l_2, l_3)^T$ can be calculated by:
$$l=p_1 \times p_2$$
We can get the transformation matrix:
$$\begin{pmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
l_1 & l_2 & l_3\\
\end{pmatrix}
$$
The algorithm is mainly implemented in the function compute_affine_H.
tiles5:
| Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image |
|---|---|---|---|
|
 |
 |
 |
facade:
| Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image |
|---|---|---|---|
|
 |
 |
 |
book1:
| Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image |
|---|---|---|---|
|
 |
 |
 |
mychess2:
| Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image |
|---|---|---|---|
|
 |
 |
 |
myfacade2:
| Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image |
|---|---|---|---|
|
 |
 |
 |
Q2.2 Evaluating angles and visualization¶
tiles5:
| Before | After |
|---|---|
| 0.167718 | -0.008091 |
| 0.034982 | -0.029306 |
| Test lines on Input Image | Test lines on Metric-Rectified Image |
|---|---|
 |
 |
facade:
| Before | After |
|---|---|
| -0.191870 | 0.046071 |
| -0.106548 | 0.010746 |
| Test lines on Input Image | Test lines on Metric-Rectified Image |
|---|---|
 |
 |
book1:
| Before | After |
|---|---|
| -0.118708 | 0.028819 |
| 0.160230 | -0.016234 |
| Test lines on Input Image | Test lines on Metric-Rectified Image |
|---|---|
 |
 |
mychess2:
| Before | After |
|---|---|
| 0.689246 | -0.010429 |
| 0.701872 | -0.135626 |
| Test lines on Input Image | Test lines on Metric-Rectified Image |
|---|---|
 |
 |
myfacade2:
| Before | After |
|---|---|
| -0.373505 | -0.516467 |
| -0.322152 | -0.499318 |
| Test lines on Input Image | Test lines on Metric-Rectified Image |
|---|---|
 |
 |
Q2.3 Brief description of the implementation¶
Based on the result in Q1, we can transform the annotated points with the result $H_{affine}$. $$x' = H_{affine}x$$ Same as Q1, we can use the cross product to get 4 lines with the annotated points in homogeneous coordinates.
For metric rectification, we have: $$l'^{\top} \mathbf{C}_{\infty}^{*'} \mathbf{m}'=0$$ Since the image has been affinely rectified, $$\mathbf{C}_{\infty}^{*'} = \begin{bmatrix} a & b/2 & d/2 \\ b/2 & c & e/2 \\ d/2 & e/2 & f \\ \end{bmatrix} $$ $d=e=f=0$
Given two pairs of perpendicular lines $l', m'$, we can solve $\mathbf{C}_{\infty}^{*'}$ by: $$(l'_1, l'_2)S(m'_1, m'_2)^T=0$$ $$(l'_1m'_1, l'_1m'_2+l'_2m'_1, l'_2m'_2)\mathbf{s}=0$$ where $\mathbf{s}=(s_{11},s_{12},s_{22})^T$
Solving this equation, we can get $$\mathbf{C}_{\infty}^{*'} = \begin{bmatrix} s_{11} & s_{12} & 0 \\ s_{12} & s_{22} & 0 \\ 0 & 0 & 0 \\ \end{bmatrix} $$
Then, we can compute the transformation matrix $H_{metric}$.
$$\mathbf{C}_{\infty}^{*'} = USU^T$$
Computing the SVD of $\mathbf{C}_{\infty}^{*'}$, we can get:
$$H_{metric}=
\begin{bmatrix}
\sqrt{\sigma_1^{-1}} & 0 & 0 \\
0 & \sqrt{\sigma_2^{-1}} & 0 \\
0 & 0 & 1 \\
\end{bmatrix}U^T
$$
The algorithm is implemented in the function compute_metric_H.
desk:
| Normal Image | Perspective Image | Annotated corners in Perspective Image | Warped and Overlaid Image |
|---|---|---|---|
 |
 |
 |
 |
mydesk:
| Normal Image | Perspective Image | Annotated corners in Perspective Image | Warped and Overlaid Image |
|---|---|---|---|
 |
 |
 |
 |
Q3.2 Brief description of the implementation¶
$\mathbf{x}_1$ and $\mathbf{x}_2$ are two sets of points from two camera views, where $$ \mathbf{x}^i_1 = \begin{pmatrix} x^i_1\\ y^i_1\\ 1\\ \end{pmatrix}, \mathbf{x}^i_2 = \begin{pmatrix} x^i_2\\ y^i_2\\ 1\\ \end{pmatrix}. $$
$\mathbf{H}$ is a $3 \times 3$ matrix. $H = \begin{pmatrix} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \\ \end{pmatrix}. $
According to $\mathbf{x}_2 = \mathbf{H}\mathbf{x}_1$: $$ \begin{pmatrix} x^i_2\\ y^i_2\\ 1\\ \end{pmatrix} = \begin{pmatrix} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \\ \end{pmatrix} \begin{pmatrix} x^i_1\\ y^i_1\\ 1\\ \end{pmatrix} = \begin{pmatrix} h_{11}x^i_1 + h_{12}y^i_1 + h_{13} \\ h_{21}x^i_1 + h_{22}y^i_1 + h_{23} \\ h_{31}x^i_1 + h_{32}y^i_1 + h_{33} \\ \end{pmatrix} $$
It can be written as: $$\mathbf{A}_i \mathbf{h}=0$$ where $$\mathbf{A}_i = \begin{pmatrix} -x^i_1 & -y^i_1 & -1 & 0 & 0 & 0 & x^i_1x^i_2 & y^i_1x^i_2 & x^i_2 \\ 0 & 0 & 0 & -x^i_1 & -y^i_1 & -1 & x^i_1y^i_2 & y^i_1y^i_2 & y^i_2 \\ \end{pmatrix} $$ We compute the SVD of $\mathbf{A}_i$ and the homography matrix $H$ is the last column of $V^T$.