16-822 Assignment 1: Projective Geometry and Homography

Q1: Affine Rectification (30 points)

1. Visual results

Given Photos

Name Input Image Annotated parallel lines on input image Affine-Rectified Image
tiles3 tiles3 anno_tiles3 rec_tiles3
tiles5 tiles5 anno_tiles5 rec_tiles5
facade facade anno_facade rec_facade

My Photos

Name Input Image Annotated parallel lines on input image Affine-Rectified Image
Court court anno_court rec_court
Board board anno_board rec_board

2. Evaluate Angles

  • tile3:
Before After
0.9928 0.9999
0.9958 0.9995
Test lines on Input Image Test lines on Affine-Rectified Image
test_tiles3 rec_anno_tiles3
  • tile5:
Before After
0.9868 0.9999
0.9987 0.9999
Test lines on Input Image Test lines on Affine-Rectified Image
test_tiles5 rec_anno_tiles5
  • facade:
Before After
0.7842 0.9999
0.9999 0.9999
Test lines on Input Image Test lines on Affine-Rectified Image
test_facade rec_anno_facade
  • board:
Before After
0.8211 0.9987
0.9999 0.9999
Test lines on Input Image Test lines on Affine-Rectified Image
test_board rec_anno_board
  • court:
Before After
0.7737 0.9993
0.9999 0.9999
Test lines on Input Image Test lines on Affine-Rectified Image
test_court rec_anno_court

3. Brief introduction of implementation

I first use pairs of points to calculate the lines using the cross product l = p1 × p2. Next, I use pairs of parallel lines to calculate the vanishing points (the intersection of the corresponding parallel lines) v = l1 × l2.

These vanishing points correspond to the directions of parallel lines in the projective plane. Using two vanishing points, we calculate the line at infinity as: l∞ = v1 × v2.

This line at infinity represents the locus of all points at infinity in the rectified image. Finally, I replace the last row of the identity matrix I with the line at infinity, normalizing it to derive the affine rectification matrix. (H = I, H[2] = l∞, H = H/H[2,2])

Q2: Metric Rectification (40 points)

1. Visual results

Given Photos

Name Input Image Annotated perpendicular lines on input image Annotated perpendicular lines on Affine-Rectified Image Rectified Image
tiles3 p2_tiles3 p2_anno_tiles3 p2_rec_tiles3 p2_metric_rec_tiles3
tiles5 p2_tiles5 p2_anno_tiles5 p2_rec_tiles5 p2_metric_rec_tiles5
facade p2_facade p2_anno_facade p2_rec_facade p2_metric_rec_facade

My Photos

Name Input Image Annotated perpendicular lines on input image Annotated perpendicular lines on Affine-Rectified Image Rectified Image
Court court p2_anno_court p2_rec_court p2_metric_rec_court
Board p2_board p2_anno_board p2_rec_board p2_metric_rec_board

2. Evaluate Angles

  • tile3:
Before After
0.1466 -0.0193
0.3993 0.0295
Test lines on Input Image Test lines on Affine-Rectified Image
p2_test_tiles3 p2_test_anno_tiles3
  • tile5:
Before After
0.1677 0.0080
0.0350 0.0293
Test lines on Input Image Test lines on Affine-Rectified Image
p2_test_tiles5 p2_test_anno_tiles5
  • facade:
Before After
-0.1918 -0.0460
-0.1065 0.0107
Test lines on Input Image Test lines on Affine-Rectified Image
p2_test_facade p2_test_anno_facade
  • board:
Before After
0.1839 -0.0391
0.2577 -0.1094
Test lines on Input Image Test lines on Affine-Rectified Image
p2_test_board p2_test_anno_board
  • court:
Before After
-0.3183 -0.0933
0.3465 -0.0707
Test lines on Input Image Test lines on Affine-Rectified Image
p2_test_court p2_test_anno_court

3. Brief introduction of implementation

I first calculate the affine rectification matrix Haffine according to Q1. Then, we utilize the fundamental constraint given by the equation l′TCinf*′m′ = 0.

Given that the image has already undergone affine rectification, the matrix Cinf*′ is simplified and defined as: $C_{\text{inf}}^{*^{\prime}} = \begin{bmatrix} a & b & 0 \\ b & c & 0 \\ 0 & 0 & 0 \end{bmatrix}$ . This formulation allows for the calculation of the conic dual to circular points, leveraging two pairs of orthogonal lines. From each pair of orthogonal lines, the following equation can be derived:

(l0⋅m0)a + (l0⋅m1+l1⋅m0)b + (l1⋅m1)c = 0

By solving this equation, we obtain the coefficients a, b and c, which define the conic dual to the circular points Cinf*′.

Then, we first conduct the singular value decomposition (SVD) of Cinf*′. The metric rectification matrix Hmetric is then formulated as follows:

$$ H_{\text{metric}} = \begin{bmatrix} \sqrt{\sigma_1^{-1}} & 0 & 0 \\ 0 & \sqrt{\sigma_2^{-1}} & 0 \\ 0 & 0 & 1 \end{bmatrix} U^T $$ Finally, we have H = HmetricHaffine.

Q3: Planar Homography from Point Correspondences (30 points)

1. Visual results

Name Normal Image Perspective Image Annotated corners in Perspective Image Warped and Overlaid Image
book desk-normal desk-perspective target_desk-perspective composite_desk-perspective
film source_table table target_table composite_table

2. Brief introduction of implementation

I selected four corners as the corresponding points as they are easy to identify. The point correspondences in source and target image satisfy x′ = Hx. Therefore, x′ × Hx = 0 gives two constraints on homography matrix H from each correspondence points, constructing two rows of matrix A as

$$ A=\left[\begin{array}{ccccccccc}-x_i & -y_i & -1 & 0 & 0 & 0 & x_i x_i^{\prime} & y_i x_i^{\prime} & x_i^{\prime} \\ 0 & 0 & 0 & -x_i & -y_i & -1 & x_i y_i^{\prime} & y_i y_i^{\prime} & y_i^{\prime}\end{array}\right] $$

Then, we can use 4 correspondence in total to solve the linear equations Ah = 0 (using SVD), where A is a matrix of size 2𝑛 × 9 and h is a vector of size 9 × 1.

Q4: Bonus: Metric Rectification from Perpendicular Lines (10 points)

1. Results

  • tile5
Input Image Annotated parallel lines on input image Affine-Rectified Image
tiles5 anno_tiles5 metric_rec_anno_tiles5
Before After
-0.0614 0.0219
0.0437 0.0095
  • chinese chess
Input Image Annotated parallel lines on input image Affine-Rectified Image
chess_cn anno_chess_cn metric_rec_anno_chess_cn
Before After
0.1081 0.0286
-0.1044 -0.0468

2. Brief introduction of implementation

Similar to Q2, we utilize the fundamental constraint given by the equation l′TCinf*′m′ = 0. The matrix Cinf*′ is now defined as $$ C_{\text{inf}}^{*^{\prime}} = \begin{bmatrix} a & b & d \\ b & c & e \\ d & e & f \end{bmatrix} $$

From each pair of orthogonal lines, the following equation can be derived:

(l0⋅m0)a + (l0⋅m1+l1⋅m0)b + (l1⋅m1)c + (l0⋅m2+l2⋅m0)d + (l2⋅m1+l1⋅m2)e + (l2⋅m2)f = 0

By solving this equation, we obtain the coefficients (a,b,c,d,e,f), which define the conic dual to the circular points Cinf*′.

Then, we first conduct the singular value decomposition (SVD) of Cinf*′. The metric rectification matrix Hmetric is then formulated as follows:

$$ H_{\text{metric}} = \begin{bmatrix} \sqrt{\sigma_1^{-1}} & 0 & 0 \\ 0 & \sqrt{\sigma_2^{-1}} & 0 \\ 0 & 0 & 1 \end{bmatrix} U^T $$

Finally, we have H = HmetricHaffine.

Q5: Bonus: More Planar Homography from Point Correspondences (10 points)

Let’s construct a ‘Quintin Tarantino’ theme time square!

1. Input Images

  • Source Images
Input Image List Pulp Fiction Kill Bill Once Upon A Time In Hollywood The Hateful Eight Django Unchained
Quintin Tarantino source_pulp killbill1 source_hollywood source_eight source_django
  • Target Image
target_pulp

2. Output Images

composite_django

3. Brief introduction of implementation

Similar to Q3, I process the above 5 film poster sequentially. By selecting the billboard corners and finding the homography between source images and target surfaces, the film posters are pasted to time square billboards one by one. In each step, I save the composite image generated in each step and use it as the target image in next step. After 5 iterations, a ‘Quintin Tarantino’ theme time square is formed.