HW1: Projective Geometry and Homography

Colleen Que

Q1: Affine Rectification

Input Image	Annotated Parallel lines on Input Image	Affine Rectified Image

| | | |

3. Evaluate Angles:

Image Name	Before	After
Facade	0.77395729920	0.9999605188736062
Facade	0.99998182	0.999972
Checker1	0.8958801150	0.9999935665
Checker1	0.9572630988	0.9999728071273944
Chess1	0.98757636192762	0.99983730958780
Chess1	0.9844509058721	0.99999695749
Keyboard	0.997760884	0.999986295
Keyboard	-0.996977056	-0.999903993367
Shelf	0.8954254281415592	0.9999840448
Shelf	0.999715098543507	0.9996144383

Description:

Calculated lines from finding the cross product of the annotated points.

 line1 = np.cross([point1[0], point1[1], 1], [point2[0], point2[1], 1])
     line2 = np.cross([point3[0], point3[1], 1], [point4[0], point4[1], 1])

Found vanishing points by taking the cross product of two lines.
```
vanishing_point = np.cross(line1, line2)
```
Took the cross product of vanishing points to find the line at infinity and used this to construct the homography matrix H.

$$H = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ l_1 & l_2 & l_3 \end{bmatrix}$$

    v1 = utils.normalize(v1)
    v2 = utils.normalize(v2)
    line_at_infinity = np.cross(v1, v2)

    # normalize
    line_at_infinity = line_at_infinity / line_at_infinity[2]


    H_affine = np.array([[1, 0, 0],
                        [0, 1, 0],
                        [line_at_infinity[0], line_at_infinity[1], line_at_infinity[2]]])

Q2: Metric Rectification

Input Image	Annotated Perpendicular Lines on Input Image	Annotated Perpendicular Lines on Affine-Rectified Image	Rectified Image

| | | | |

3. Evaluate Angles:

Image Name	Before	After
Facade	-0.6664966857689815	0.0503733122293811
Facade	0.6902846557329554	0.0126798416894802
Checker1	-0.8979851319218762	0.0043526501625
Checker1	0.790050666869242	0.00604931080793
Chess1	0.705015942	-0.021783689655921312
Chess1	-0.57054451917898	-0.011004444616700
Keyboard	0.817811623134940	0.074495275
Keyboard	0.7293617798571533	-0.038503035
Shelf	-0.912905	-0.05763805
Shelf	-0.9023554956443	0.032474856

Description:

Finding C

$$ l^{'T} C_{\infty}^{*'} m' = 0 \text{ since } l' \text{ and } m' \text{ are perpendicular.}\\ \text{This can be expressed as:} \\ (l_1' \, l_2' \, l_3') \begin{bmatrix} K K^T & 0 \\ 0^T & 0 \end{bmatrix} \begin{bmatrix} m_1' \\ m_2' \\ m_3' \end{bmatrix}\\ $$

$$\text{After normalization, this can be rewritten as:}$$

$$ \begin{pmatrix} l_1 \\ l_2 \end{pmatrix}^T S \begin{pmatrix} m_1 \\ m_2 \end{pmatrix} = 0 \\ $$

$$ (l_1 m_1 + l_1 m_2 + l_2 m_1 + l_2 m_2) s = 0 $$

In Code:


   A = np.array([[l1[0] * m1[0], l1[0]*m1[1] + l1[1]*m1[0], l1[1]*m1[1]],
                  [l2[0] * m2[0], l2[0]*m2[1] + l2[1]*m2[0], l2[1]*m2[1]]
                  ])

And find the null space of A to find s and thus C:

    Vt = np.linalg.svd(A)[-1]
    s1, s2, s3 = Vt[-1]

    #dual conic
    C = np.array([[s1, s2,0], [s2, s3,0],[0,0,0]])

$$ \text{C can be decomposed as: } U \begin{pmatrix} \sigma _1 &0&0\\ 0&\sigma _2 & 0\\ 0&0&0 \end{pmatrix} U^T \\ \text{And } $$

$$ H = U \begin{pmatrix} \sqrt{\sigma_ 1^{-1} }&0&0\\ 0&\sqrt{\sigma_ 2 ^{-1}}& 0\\ 0&0&0 \end{pmatrix} U^T $$

As Code


    U,S,Vt_C = np.linalg.svd(C)

    inverse_sqrt = np.where(S > 0, np.sqrt(1 / S), 1)
    Sigma_inv = np.diag(inverse_sqrt)

    H2 = np.dot(Sigma_inv,Vt_C)

Q3: Planar Homography from Point Correspondences

Normal Image	Perspective Image	Annotated Corners in Perspective Image	Warped and Overlaid Image

| | | | |

Description :

Constructed an 8x9 matrix A to solve the equation ( Ah = 0 ).

$$ Ah = 0 $$

$$ \begin{pmatrix} -x_{i_2} & -y_{i_2} & -1 & 0 & 0 & 0 & x_{i_1} x_{i_2} & x_{i_1} y_{i_2} & x_{i_1} \\ 0 & 0 & 0 & -x_{i_2} & -y_{i_2} & -1 & y_{i_1} x_{i_2} & y_{i_1} y_{i_2} & y_{i_1} \end{pmatrix} \begin{pmatrix} h_1 \\ h_2 \\ h_3 \\ h_4 \\ h_5 \\ h_6 \\ h_7 \\ h_8 \\ h_9 \end{pmatrix} = 0 $$

As Code:

    a = np.array([])

    for i, e in enumerate(x1):
        xi_1 = e[0]
        yi_1 = e[1]
        xi_2 = x2[i][0]
        yi_2 = x2[i][1]
        ai_1 = np.array([-xi_2, -yi_2, -1, 0, 0, 0,
                         xi_1 * xi_2, xi_1*yi_2, xi_1])
        ai_2 = np.array([0, 0, 0, -xi_2, -yi_2, -1,
                        yi_1*xi_2, yi_1*yi_2, yi_1])

        if a.size == 0:
            a = ai_1
            a = np.vstack((a, ai_2))
        else:
            a = np.vstack((a, ai_1))
            a = np.vstack((a, ai_2))

h is in the null space of A and corresponds to the nine values of the homography matrix.

$$ H = \begin{pmatrix} h_1 & h_2 & h_3 \\ h_4 & h_5 & h_6 \\ h_7 & h_8 & h_9 \end{pmatrix}$$

    vh = np.linalg.svd(a)[-1]
    h = vh[-1]

    Homography = h.reshape(3, 3)
    return Homography

Q5: Bonus: More Planar Homography from Point Correspondences

Input Images:

Output Image:

Brief Description:

Utilized homography from part 3; used a for loop to sample individual homographies for each book cover and mapped to the labeled points on the times square image.