Cosine values

before: 0.9733956073791686 | after: 1.0

before: 0.9856917939856646 | after: 1.0

before: 0.8964448119431402 | after: 0.9999999999999998

before: 0.9574522354365923 | after: 0.9999729375269912

Cosine values

before: 0.9375299616805701 | after: 1.0

before: 0.9999770229869747 | after: 1.0

before: 0.9868327646462675 | after: 0.9999251970022869

before: 0.9987191070813485 | after: 0.9999383464088627

Cosine values

before: 0.9999289183728965 | after: 1.0

before: 0.9334215830812338 | after: 1.0

before: 0.999989581438351 | after: 0.9999934408698984

before: 0.9599844125720752 | after: 0.9952989626474258

Cosine values

before: 0.9995347698277122 | after: 0.9999972902297625

before: 0.9991796747215933 | after: 0.9999999670411781

First we obtain the annotated points. For affine rectification, we require atleast a 2 pairs of parallel lines (i.e. 8 lines) to compute the vanishing point. Once we have the vanishing point, we can compute the homography matrix using the line at infinity.
Key equations:
1. Computing lines from points: l = cross(p1, p2)
2. vanishing point computation: vp = cross(l1, l2)
3. line at infinity: l_inf = cross(vp1, vp2) = (l1, l2, l3)
4. homography matrix H = [1 0 0; 0 1 0; l_inf(0) l_inf(1) l_inf(2)]
Note: We first need to convert the points to homogeneous coordinates before computing the cross product.

Cosine values

before: 0.004750540219098102 | after: 0.002368762375672669

before: -0.06293803653600322 | after: -0.01994429172376085

before: 0.1677181548716633 | after: -0.005337364178073585

before: 0.03498266075814056 | after: -0.009135920402140988

Cosine values

before: 0.027207666007229535 | after: 0.004063801160690535

before: -0.2560695706782867 | after: -0.0006417132199867043

before: -0.25267090098365785 | after: 0.0012669929907758112

before: 0.08817394212374802 | after: 0.0041273270781477306

Cosine values

before: 0.18494237312262277 | after: -0.02027509305324703

before: -0.6935565095593584 | after: -0.014737614642025144

before: -0.1187084254612817 | after: 0.003399562397940435

before: 0.1602306433136178 | after: -0.03443089375088782

Cosine values

before: 0.41620461310910956 | after: -0.012797361949143254

before: -0.046597782869751844 | after: 7.464054267437305e-05

before: 0.6212890010687858 | after: 0.06521993259895495

First we obtain the annotated points of perpendicular. For metric rectification, we require atleast a 3 pairs of perpendicular lines (i.e. 6 lines) to compute the metric recitification matrix from a affine recitified image.

Procedure:
1. First we apply the affine rectification to the image.
2. We apply the affine transformation to the annotated perpendicular points.
3. Compute the perpendicular lines in the affine rectified image.
4. Create a matrix A with the transformed annontated perpendicular lines.
5. Compute the SVD of A and extract the last column of V. This represents the conic at infinity in the rectified image, C_inf_star
6. Compute the SVD of C_inf_star, C_inf_star = U @ S @ Ut.
7. S = [sigma1 0 0; 0 sigma2 0; 0 0 0] => Metric rectification matrix M = [inv(sqrt(sigma1)) 0 0; 0 inv(sqrt(sigma2)) 0; 0 0 1] @ Ut

Key equations:
1. Conic at infinity, C_inf_star = [a b/2 d/2; b/2 c e/2; d/2 e/2 f]
2. If l = [l1 l2 l3] is a line1 and m = [m1, m2, m3] is the perpendicular line 2, then l.T @ C_inf_star @ m = 0
3. Stacking all the perpendicular lines, we get A @ c = 0
where c = [c1 c2 c3] where c1 = a, c2 = b/2, c3 = c.
4. Each row of A will be [l1*m1, l1*m2 + m1*l2, l2*m2]. Stack up all the rows to get A.
5. SVD of A will provide c = [c1 c2 c3] which forms the conic at infinity.

Note:
Since we first affine rectify the image, the d,e,f values of the C_inf_star are equal to 0. This leaves us with a,b,c values of the C_inf_star matrix. So we need three constraints to extract C_inf_star.
The constraints are: l'.T @ C_inf_star @ l = 0 for all the three pairs

For computing the homography between two images, we need a minimum of 4 correspondences.

Procedure:
1. Normalise all the annotated points.
2. Create a matrix A based on the homography constraints imposed by correspondences between the images.
3. Compute the SVD of the matrix A, the last column of V is the flattened form of the required homography matrix.
4. Reshape the flattened homography matrix to get the 3x3 homography matrix.
5. Compute the homography for the originl image by de-normalising the homography matrix.
6. Apply the homography matrix to the perspective image to get the warped image.
7. Overlay the warped image on the normal image using masking.

Key equations:
1. Transformation matrix for normalisation: T = [1/w 0 -0.5; 0 1/h -0.5; 0 0 1]
2. Normalised points: p' = T @ p
3. For a given correspondence, x' = [x' y' 1] and x = [x y 1], the constraints on H are:
[0 0 0 -x -y -1 y'x y'y y'] @ h = 0
[x y 1 0 0 0 -x'x -x'y -x'] @ h = 0
4. A = [0 0 0 -x1 -y1 -1 y1x1 y1y1 y1; x1 y1 1 0 0 0 -x1x1 -x1y1 -x1; ...]
4. SVD of A: A = U @ S @ Vt. h = V[:,-1]
5. H = T_tgt_inv @ h @ T_src

Cosine values

before: 0.027207666007229535 | after: 0.013018928201640544

before: 0.08817394212374802 | after: -0.0004990418256242032

before: 0.1038655520376901 | after: -0.0057830243445201575

For computing the metric recitification dirctly from the perspective image, we need a minimum of 5 perpendicular line pairs.

Procedure:
1. Compute the perpendicular lines in the affine rectified image.
2. Create a matrix A with the transformed annontated perpendicular lines.
3. Compute the SVD of A and extract the last column of V. This represents the conic at infinity in the rectified image, C_inf_star
4. Compute the SVD of C_inf_star, C_inf_star = U @ S @ Ut.
5. S = [sigma1 0 0; 0 sigma2 0; 0 0 0] => Metric rectification matrix M = [inv(sqrt(sigma1)) 0 0; 0 inv(sqrt(sigma2)) 0; 0 0 1] @ Ut

Key equations:
1. Conic at infinity, C_inf_star = [a b/2 d/2; b/2 c e/2; d/2 e/2 f]
2. If l = [l1 l2 l3] is a line1 and m = [m1, m2, m3] is the perpendicular line 2, then l.T @ C_inf_star @ m = 0
3. Stacking all the perpendicular lines, we get A @ c = 0
where c = [c1 c2 c3 c4 c5 c6] where c1 = a, c2 = b/2, c3 = c, c4 = d/2, c5 = e/2, c6 = f.
4. Each row of A will be [l1*m1, l1*m2 + m1*l2, l2*m2, l1*m3+m1*l3, l2*m3+m2*l3, l3*m3]. Stack up all the rows to get A.
5. SVD of A will provide c = [c1 c2 c3 c4 c5 c6] which forms the conic at infinity.

Note:
1. Since we are not doing the affine rectification, all the cooefficients of the C_inf_star matrix are non-zero.
2. Therefore we need a minimum of 5 constraints to compute the metric rectification (1dof removed due to scale invariance).