HW1: Projective Geometry and Homography

Late day image:

---

Q1: Affine Rectification (30 points)

Your goal in this question is to generate affinely correct warps for images that are captured through perspective cameras (assume pinhole camera model), with annotations of at least 2 pairs of parallel lines.

Submission

1. Input Images

2. Output Images: Rectified images and intermediate images with color coded annotation that you used. See the following figures:

3. Evaluate Angles (more precisely, the cosines) of at least another 2 pairs of parallel lines before and after rectification. This will show how far from 1 the cosines are in the original image and how close to correct the final affine rectification is.

Input Image	Annotated Parallel Lines on Input Image	Affine-Rectified Image	Test Lines on Affine-Rectified Image	Cosine of Angle 1 (Before)	Cosine of Angle 1 (After)	Cosine of Angle 2 (Before)	Cosine of Angle 2 (After)
				0.9334215830812338	0.9927955506343918	0.999989581438351	0.9999861955705955
				0.9856917939856645	0.9999952070698461	0.8964448119431402	0.9999946539884772
				0.9634361913316474	0.9999944597780334	0.9878470787639924	0.999769305300229
				0.7842324298864692	0.9999781395179717	0.999997015205378	0.9999869448826936
				0.9644476218485872	0.9632522676018825	0.9071546562014172	0.9921373798782696
				0.9999962841172901	0.9998647579803845	0.9744650898435482	0.9999223615538647

4. Brief description of your implementation (i.e., the algorithm followed with relevant equations and what annotations were used).

Algorithm Steps:

1. Annotate Parallel Lines:

   • Input: Two pairs of parallel lines are annotated in the image, represented by their endpoints. Let these parallel lines be:
     • First pair: $ (x_1, y_1), (x_2, y_2) $ and $ (x_3, y_3), (x_4, y_4) $,
     • Second pair: $ (x_5, y_5), (x_6, y_6) $ and $ (x_7, y_7), (x_8, y_8) $.

2. Compute Vanishing Points: The vanishing point for each pair of parallel lines is computed as the intersection of their extensions.

   • Mathematical Formulation:
     • Each line is represented in homogeneous coordinates: $ \mathbf{l_1} = (x_1, y_1, 1) \times (x_2, y_2, 1) $ $ \mathbf{l_2} = (x_3, y_3, 1) \times (x_4, y_4, 1) $
     • The vanishing point is the intersection of these two lines: $ \mathbf{v_1} = \mathbf{l_1} \times \mathbf{l_2} $
     • Normalize the vanishing point to obtain $ (v*{x}, v*{y}) $ by dividing by the homogeneous coordinate: $ \mathbf{v_1} = \left(\frac{v_{x}}{v_z}, \frac{v_{y}}{v_z}\right) $
   • Result: Two vanishing points are calculated: $ \mathbf{v_1} $ from the first pair of parallel lines and $ \mathbf{v_2} $ from the second pair.

3. Construct Affine Homography Matrix:

   • Affine Correction: remove perspective distortion by mapping the two vanishing points to "infinity,"
   • The affine homography matrix $ \mathbf{H} $ is constructed based on the vanishing points $ \mathbf{v_1} $ and $ \mathbf{v_2} $.
   • Homography Matrix: $ \mathbf{H} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ v_{1x} - v_{2x} & v_{1y} - v_{2y} & 1 \end{pmatrix} $
   • This matrix warps the image such that the perspective effect is removed, and lines that were parallel in the real world remain parallel in the warped image.

4. Apply Affine Warp:

   • Warping the Image: Using OpenCV’s cv2.warpPerspective(), the affine homography matrix is applied to the input image to generate the affine-corrected output.

Q2: Metric Rectification (40 points)

Your goal in this question is to generate metrically correct warps for images that are captured through perspective cameras (assume pinhole camera model), with annotations of at least 2 pairs of perpendicular lines, based on your result from Q1.

Submission

1. Input Images
2. Output Images: Rectified images and intermediate images with color coded annotation that you used. See the following figures:
3. Evaluate Angles (more precisely, the cosines) of at least another 2 pairs of perpendicular lines before and after rectification. This will show how far from 0 the cosines are in the original image and how close to correct the final metric rectification is. Check the following Table and Figure as an example:

Input Image	Annotated perpendicular lines on input image	Annotated perpendicular lines on Affine-Rectified Image	Test lines on Affine-Rectified Image	Cosine Angle 1 Before	Cosine Angle 1 After	Cosine Angle 2 Before	Cosine Angle 2 After
				0.18494237312262282	-0.039220646448387846	0.1602306433136178	-0.053188051701753426
				-0.10157620715787374	0.03475471106538331	0.3993378380341757	0.288512318208863
				-0.06293803653600322	0.042247690978592184	0.03498266075814055	-0.06753984399109939
				0.876248063857186	0.827636622616270	0.4487142399450566	0.4031523026598513

| | | | |0.13100162936447854 | -0.05237211670264567 | 0.09392259459566803| -0.08944917821863532

4. Brief description of your implementation (i.e., the algorithm followed with relevant equations and what annotations were used).

Computing Metric Homography Matrix:

• The function computes the metric homography matrix using two pairs of annotated parallel lines.

• It constructs the conic matrix from the line pairs, solves a linear system, and then uses Singular Value Decomposition (SVD) to construct the final homography matrix. Get $H=1/\sqrt{\Sigma^{-1}} U^T$ where $H$ is the homography matrix, $\Sigma$ is the diagonalized matrix, and $U$ is the left singular matrix.

  • Equations involved:
    • Line equations are computed using the cross product of points: $ l = p1 \times p2 $
    • The linear system is constructed as: $ A \cdot x = b $ where $A $is constructed from the line equations.
    • SVD is applied to decompose the conic matrix into singular values, and the diagonalized matrix is used to compute the metric homography.

• After computing the affine homography from a previous stage (saved from "q1"), apply this affine transformation

• The affine-transformed image is then passed through the metric homography transformation to obtain the metrically rectified image

• The homography is applied using cv2.warpPerspective().

Q3: Planar Homography from Point Correspondences (30 points)

Your goal in this question is to estimate homographies between two images using point correspondences.

Submission

1. Input Images
2. Output Images: Warped images and intermediate images with annotation that you used (i.e., corners/point correspondences). See the following figures:

Normal Image	Perspective Image	Annotated corners in Perspective Image	Warped and Overlaid Image

3. Biref description of your implementation (i.e., the algorithm followed with relevant equations and what annotations were used).

The task of the implementation is to warp and overlay one image onto another using homography transformation based on annotated corresponding points.

1. Annotation Loading:

   • The algorithm starts by reading annotated points from both the source and destination images. These points are used to define correspondences between the source and destination image planes.
   • The annotations consist of four points that define the corners of an object or region of interest.

2. Homography Matrix Calculation:

   • The homography matrix $ H $ is a $ 3 \times 3 $ matrix that maps points from the source image plane to the destination image plane, using the relationship: $ \mathbf{p'} = H \mathbf{p} $ where:

     • $ \mathbf{p} = [x, y, 1]^T $ is a point in the source image.
     • $ \mathbf{p'} = [u, v, 1]^T $ is the corresponding point in the destination image.
     • $ H $ is the homography matrix that transforms the coordinates of $ \mathbf{p} $ to match the perspective of $ \mathbf{p'} $.

   • The homography matrix is computed by solving a system of linear equations derived from the correspondences of four points between the source and destination image: $ \begin{aligned} -x_1 h_{31} - y_1 h_{32} - h_{33} + u_1 (x_1 h_{11} + y_1 h_{12} + h_{13}) &= 0 \\ -x_1 h_{31} - y_1 h_{32} - h_{33} + v_1 (x_1 h_{21} + y_1 h_{22} + h_{23}) &= 0 \end{aligned} $

     • This system is solved using SVD, yielding the homography matrix $ H $.

3. Applying Homography:

   • using the transformation: $ \mathbf{p'} = H \mathbf{p} $ for each pixel in the source image, mapping it to the destination image space.
   • The warped image is then combined with the destination image, replacing or overlaying parts of the destination image where the warped source image lands.

Q4: Bonus: Metric Rectification from Perpendicular Lines (10 points)

In Q2, we generate metrically correct warps of images with annotations of 2 pairs of perpendicular lines based on the result from Q1. In this question, the goal is to metric-rectify the image directly from at least 5 pairs of perpendicular lines. Note you can use more than 5 pairs to get more accurate results.

bmission**

1. Input Images
2. Output Images: Rectified images and intermediate images with color coded annotation that you used. Annotate the lines that were used. See the following figure:
3. Angles (more precisely, the cosines) of at least 3 pairs of perpendicular lines before and after rectification.

Input Image	Annotated perpendicular lines	Rectified Image	Cosine Angles Before	Cosine Angles After
			[0.15429748144559605, 0.1908450204198642, -0.04421100757519586, 0.09347394725534008, 0.17101841325527012]	[0.09334885835784874, 0.1353921197336838, 0.16266338373734954, -0.11421067660502472, 0.16340038805539955]
			[0.0730526939324927, 0.09618219734203873, 0.09873471740774624, 0.06359050720253223, -0.12717541693502443]	[-0.08194052251863142, 0.24609934416462442, -0.05625177050653589, 0.214571722264276, 0.027486993163106585]

4. Biref description of your implementation.

• Annotate 5 pairs of perpendicular lines in the image.
• Use each pair of perpendicular lines to get one linear constraint.
• Construct a linear system of equations using the constraints.
• Solve the linear system using SVD to get the metric rectification homography.
• Apply the homography to the image using cv2.warpPerspective().
• Evaluate the cosines of the angles of the perpendicular lines before and after rectification.

Q5: Bonus: More Planar Homography from Point Correspondences (10 points)

Try to be creative and do something fun!

Overlay multiple normal images onto the perspective image using homography transformations.

Implementatioin:

• Annotate corresponding points in the perspective images.
• Compute the homography matrix using the annotated points between the normal and perspective images.
• Apply the homography transformation to warp each normal image onto the perspective image.
• Overlay the warped normal images onto the perspective image to create a composite image.

The result of the implementation is a composite image shows as follows: