Geometry-based Methods for Vision

Assignment 1

Aviral Agrawal (avirala)

Q1. Affine Rectification

In this question, we use 2 pair of parallel lines to perform affine rectifictation. The algorithm is as follows:
  1. Given a pair of parallel line, find their cross product to yield their intersection point: \[l_1 \times m_1 = x_1\]
  2. Using the obtained points, compute the line at infinity: \[x_1 \times x_2 = l_{\infty}\]
  3. The line at infinity has the following coefficients: \[ l_{\infty} = \begin{bmatrix} a*c \\ b*c \\ c \end{bmatrix} \]
  4. The line can also be represented as follows: \[ l_{\infty} = \begin{bmatrix} a \\ b \\ 1 \end{bmatrix} \]
  5. Using the line at infinity, we can compute the homography matrix: \[ H = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ a & b & 1 \end{bmatrix} \]
Table 1. Q1 results on provided images
Input Image Annotated parallel lines on input image Affine-Rectified Image Angles before rectification Angles after rectification
Cathedral Cathedral Cathedral -0.9878470787639924
0.9845941270659719		 -0.9997703794618689
0.9999978464776054
Cathedral Cathedral Cathedral 0.7842324298864692
0.999997015205378		 0.999978139517972
0.9999869448826935
Cathedral Cathedral Cathedral 0.9868327646462675
0.9987191070813484		 0.9999251970022867
0.999938346408863


Table 2. Q1 results on custom images
Input Image Annotated parallel lines on input image Affine-Rectified Image Angles before rectification Angles after rectification
Cathedral Cathedral Cathedral 0.9990013601615017
-0.9772162545582432		 0.9999995048776282
-0.9996257960587244
Cathedral Cathedral Cathedral 0.9279215873244215
0.9999970045944495		 0.9998456444333405
0.9999927260250941


Q2. Metric Rectification

In this question, we use 2 pair of perpendicular lines to perform metric rectifictation (given affine rectified images). The algorithm is as follows:
  1. Given a pair of perpendicular lines, \(l'\) and \(m'\): \(l' C_{\infty}^{*\prime} m' = 0\)
  2. Since the image is already affine rectified, hence we only need to use 2 pair of perpendicular lines.
  3. Next, we create \(C_{\infty}^{*\prime} = \begin{bmatrix} a & b/2 & d/2 \\ b/2 & c & e/2 \\ d/2 & e/2 & f \end{bmatrix}\)
  4. We solve the equation in point1 using SVD. We construct a matrix of form \(Ac=0\) with \(c = \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \end{bmatrix}\)
  5. Since the input images are affine rectified, so \(d = e = f = 0 \)
  6. After obtaining \(C_{\infty}^{*\prime}\), we solve \(C_{\infty}^{*} = H C_{\infty}^{*\prime} H^T\)
  7. We perform SVD on \(C_{\infty}^{*\prime}\) to obtain \(\sigma_1\) and \(\sigma_2\) and the matrix \(U\)
  8. Finally, \(H = \begin{bmatrix} \frac{1}{\sqrt{\sigma_1}} & 0 & 0 \\ 0 & \frac{1}{\sqrt{\sigma_2}} & 0 \\ 0 & 0 & 1 \end{bmatrix} \cdot U^T\)


Table 3. Q2 results on provided images
Input Image Annotated perpendicular lines on input image Annotated perpendicular lines on Affine-Rectified Image Rectified Image Angles before rectification Angles after rectification
Cathedral Cathedral Cathedral Cathedral -0.668588141024195
0.044795213899351836		 0.021182064318718833
0.009640234139806901
Cathedral Cathedral Cathedral Cathedral 0.19187095182680747
-0.10654819601370301		 -0.04607186861647671
0.01074685806717023
Cathedral Cathedral Cathedral Cathedral -0.16771815487166336
-0.0349826607581406		 0.008091866263126584
0.02930693273291497


Table 4. Q2 results on custom images
Input Image Annotated perpendicular lines on input image Annotated perpendicular lines on Affine-Rectified Image Rectified Image Angles before rectification Angles after rectification
Cathedral Cathedral Cathedral Cathedral 0.1873447740149817
0.3011911793414273		 0.09080122410137553
0.36560057838142845
Cathedral Cathedral Cathedral Cathedral 0.15842975140751553
-0.13120871902808207		 -0.005232267839201996
0.0618378541885495


Q3. Planar Homography from Point Correspondences

In this question, we use atleast 4 pair of corresponding points to perform planar homography. The algorithm is as follows:
  1. Collect Correspondences: Identify at least 4 pairs of corresponding points between two images.
  2. Set up Equations: For each pair, create two linear equations based on the homography matrix H
  3. Solve for H: Solve the system of equations (usually using SVD) to find the 3x3 homography matrix.


Table 5. Q3 results on provided image
Normal Image Perspective Image Annotated corners in Perspective Image Warped and Overlaid Image
Cathedral Cathedral Cathedral Cathedral


Table 6. Q3 results on custom image
Normal Image Perspective Image Annotated corners in Perspective Image Warped and Overlaid Image
Cathedral Cathedral Cathedral Cathedral


Q4. Bonus: Metric Rectification from Perpendicular Lines

In this question, we use atleast 5 pair of perpendicular lines to perform metric rectifictation. The algorithm is as follows:
  1. Given a pair of perpendicular lines, \(l'\) and \(m'\): \(l' C_{\infty}^{*\prime} m' = 0\)
  2. Next, we create \(C_{\infty}^{*\prime} = \begin{bmatrix} a & b/2 & d/2 \\ b/2 & c & e/2 \\ d/2 & e/2 & f \end{bmatrix}\)
  3. We solve the equation in point1 using SVD. We construct a matrix of form \(Ac=0\) with \(c = \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \end{bmatrix}\)
  4. After obtaining \(C_{\infty}^{*\prime}\), we solve \(C_{\infty}^{*} = H C_{\infty}^{*\prime} H^T\)
  5. We perform SVD on \(C_{\infty}^{*\prime}\) to obtain \(\sigma_1\) and \(\sigma_2\) and the matrix \(U\)
  6. Finally, \(H = \begin{bmatrix} \frac{1}{\sqrt{\sigma_1}} & 0 & 0 \\ 0 & \frac{1}{\sqrt{\sigma_2}} & 0 \\ 0 & 0 & 1 \end{bmatrix} \cdot U^T\)


Table 6. Q4 results on provided image
Input Image Annotated perpendicular lines Rectified Image Angles before rectification Angles after rectification
Cathedral Cathedral Cathedral -0.12463807558974306
-0.17666406279377408
0.02748623746057998		 0.028851228375302018
-0.023910065043987922
-0.06807868885760726


Table 7. Q4 results on custom image
Input Image Annotated perpendicular lines Rectified Image Angles before rectification Angles after rectification
Cathedral Cathedral Cathedral 0.30233163660652657
-0.29644213333868763
-0.48095537579617925		 0.020382214593444865
0.057245674993488635
0.033305380374653804


Q5. Bonus: More Planar Homography from Point Correspondences

In this question, we use atleast 4 pair of corresponding points to perform planar homography. The algorithm is as follows:
  1. Collect Correspondences: Identify at least 4 pairs of corresponding points between two images.
  2. Set up Equations: For each pair, create two linear equations based on the homography matrix H
  3. Solve for H: Solve the system of equations (usually using SVD) to find the 3x3 homography matrix.
Now we repeat this algorithm for multiple images and then superimpose the warped images on the base image.

Table 8. Q5 Normal images used
Cathedral Cathedral Cathedral Cathedral


Table 9. Q5 Perspective image and the warping
Perspective Image Annotated corners in Perspective Image Warped and Overlaid Image
Cathedral Cathedral Cathedral