HW1: Projective Geometry and Homography

Submitted by: Kallol Saha

Please Note: Images have been set to a specific width and height to make them fit inside the given space, some of their proportions may be a little warped. You can find the original images inside the results folder.

Q1: Affine Rectification

1. Input Images

Book Facade Tiles House Temple

2. Parallel line annotations and Affine images

Name Book Facade Tiles House Temple
Parallel lines
Affine Image

3. Evaluation of cosines after affine rectification of different set of lines

Name Book Facade Tiles House Temple
Parallel lines
Cosines Before 0.9999, 0.9599 0.7842, 0.9999 0.9868, 0.9987 0.9936, 0.9998 0.9999, 0.9991
Affine Image
Cosines After 0.9999, 0.9952 0.9999, 0.9999 0.9999, 0.9999 0.9940, 0.9990 0.9999, 0.9995

4. Description of approach

  1. Convert annotated points to homogenous points
  2. Find the line between points using cross product
  3. Find the points at infinity on image plane using cross product between lines
  4. Find the line at infinity using cross product between points at infinity
  5. Define the homography to convert the image to an affine image H:
    [1 0 0]
    [0 1 0]
    [l1 l2 l3]
    l1 l2 l3 are the elements of the line at infinity
  6. Warp the points and the image using this homography
  7. The cosines are given by the dot product between 2 lines. Note: After warping points, they need to be translated to account for any change in reference frame (xmin, ymin) caused by the homography

Q2: Metric Rectification

1. Input Images

Book Facade Tiles House Temple

2. Perpendicular line annotations and Metric rectification

Name Book Facade Tiles House Temple
Perpendicular lines
Affine Image
Metric Image

3. Evaluation of cosines after metric rectification of different set of lines

Name Book Facade Tiles House Temple
Parallel lines
Cosines Before 0.1187, -0.1602 0.1918, -0.1065 -0.1677, -0.0349 0.7428, -0.0487 0.8888, 0.2398
Affine Image
Cosines After -0.0288, 0.0162 -0.0460, 0.0107 0.0080, 0.0293 0.2284, -0.0042 0.8849, 0.1210

4. Description of approach

  1. Calculate the inverse of the affine transform and apply it to the annotated lines
  2. Convert affine transformed points to homogenous coordinates
  3. Given that the image is already affine, we can represent the conic at infinity as C =
    [a b/2 0]
    [b/2 c 0]
    [0 0 0]
    since d = e = f = 0
  4. Conic at infinity satisfies the constraint transpose(l) @ C @ u = 0, where l and u are annotated perpendicular lines
  5. This constraint can be represented as Ac = 0 (c = [a, b, c])where each row of A is represented as
    [m1*l1 ; (m2*l1 + m1*l2)/2 ; (m2*l2)]
    where the elements of lines l and m are [l1 l2 l3] and [m1 m2 m3]
    The number of rows in A is the number of pairs of annotated lines. Here, we have 2 rows since the image is already affine rectified and we need only 2 constraints
  6. Therefore, conic at infinity can be formed using the solution for a, b, c obtained from the null space of A. This is done using SVD
  7. Then, we find the singular values of the conic at infinity
  8. For a metric transformation, we can get the Homography H =
    [root(1/s1) 0 0]
    [0 root(1/s2) 0] @ transpose(U)
    [0 0 1]
    where s1 and s2 are singular values of conic at infinity C.
  9. The image and the lines are warped using the homography H, to get a metric image
  10. The cosines are given by the dot product between 2 lines. Note: After warping points, they need to be translated to account for any change in reference frame (xmin, ymin) caused by the homography

Q3: Planar Homography from Point Correspondences

1. Annotations, warping and overlaying:

Normal Image Perspective Image Annotated corners Warped and Overlaid Image

2. Description of approach

  1. Make both sets of 4 annotated points homogenous (The first set is just the corners of the normal image)
  2. Define an 8x9 matrix A. Each point correspondence gives us 2 constraints, so we have 8 constraints and 9 variables
  3. The 9 variables are the elements of the homography H (3x3 matrix) which can be found using DLT.
  4. We need to solve for X' = H @ X. This can be represented as cross_product(X', H @ X) = zero vector, so that it is invariant to scale
  5. This can be simplified to an equation Ah = 0, where h is the elements of H flattened to a column vector
  6. Each pair of consecutive rows of A =
    [0 -w*XT, y*XT]
    [w*xT, 0, -x*XT]
    where XT is a row vector of the point X and (x, y, w) are the elements of X' => Each row has 9 elements.
  7. The null space of A gives us the solution for homography H, found using SVD
  8. The normal image is warped and overlaid on top of the perspective image by using the homography H and a mask

Q4: Bonus: Metric Rectification from Perpendicular Lines (10 points)

1. Input Images

Tiles Temple

2. Perpendicular line annotations and Metric images

Tiles Temple
Perpendicular lines
Metric Image

3. Evaluation of cosines after affine rectification of different set of lines

Tiles Temple
Perpendicular lines
Cosines Before 0.4970, -0.5164, -0.2061 0.9791, 0.9407, 0.8423
Metric Image
Cosines After 0.0315, 0.0185, -0.0341 0.1345, 0.0832, 0.0652

4. Description of approach

  1. The approach is similar to question 2, but we do not convert to an affine image first
  2. Instead, we can arrange the 5 perpendicular line constraints into a 5x6 matrix A
  3. These can then be used to find the elements a,b,c,d,e,f of the conic at infinity using the null space of A from SVD
  4. We then find the singular values of the conic at infinity and the U matrix using SVD
  5. The homography H = Singular value matrix @ U.T
  6. The cosines are given by the dot product between 2 lines

Q5: Bonus: More Planar Homography from Point Correspondences

Normal Image Perspective Image Annotated corners Warped and Overlaid Image
  1. Given many point annotations on a single perspective image,
  2. Using the DLT algorithm from question 3,
  3. Re-write the output image as the new perspective image, after overlaying a normal image on top of it
  4. By doing this, we can sequentially overlay multiple images on top of the same image