Assignment 1: Projective Geometry and Homography

Q1: Affine Rectification (30 points)

The left image shows the image after annotation, and the right image shows the affine rectification result.

Before	After
0.9575	-0.9999
-0.9643	-0.9999

Before	After
0.9163	0.9999
0.9471	0.9999

Before	After
0.9704	1.0000
0.9876	0.9999

Before	After
0.9999	1.0000
0.7874	1.0000

Before	After
0.9464	1.0000
0.9996	1.0000

Before	After
0.9999	1.0000
0.4667	1.0000

Before	After
0.9862	1.0000
-0.9801	-1.0000

Algorithm Description:

Step 1: User Annotation

• The user is prompted to annotate four lines on the image, representing two pairs of parallel lines. These annotations are collected using mouse clicks on the image.
• Each line pair is drawn using two points, and the selected points are displayed on the image.

Step 2: Computing the Affine Rectification Homography

The annotated points are used to calculate the corresponding lines in homogeneous coordinates using the cross product:

$$ l = p_1 \times p_2 $$

The vanishing points for each line pair are calculated by taking the cross product of two lines:

$$ vp = l_1 \times l_2 $$

The line at infinity is derived from the two vanishing points:

$$ l_{\infty} = vp_1 \times vp_2 $$

The affine rectification homography matrix \( H_{\text{affine}} \) is constructed with the line at infinity:

$$ H_{\text{affine}} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ l_{\infty}[0] & l_{\infty}[1] & l_{\infty}[2] \end{bmatrix} $$

Step 3: Affine Rectification

The original image is warped using the computed homography matrix, resulting in a rectified image where the annotated lines appear parallel.

Usage

1. Run the script with the following command (for example below) in the q1.py file:

    python q1.py tiles5.JPG 

2. Annotate the image by clicking on four points to create two pairs of parallel lines.

3. The script will compute the affine transformation and display the rectified image.

4. The annotated image (q1_annotated.png) and the rectified image (q1_rectified_image.png) will be saved in the current directory.

Q2: Metric Rectification (40 points)

The left image shows the image after parallel annotation, middle one is perpendicular annotation, and the right image shows the affine rectification result.

Before	After
-0.1886	0.0107
0.0778	0.0400

Before	After
0.8999	0.0115
0.8963	0.0134

Before	After
0.6842	0.1392
-0.5556	-0.0625

Before	After
-0.6838	0.0000
0.6679	-0.0000

Before	After
0.6385	-0.3068
-0.6248	0.3105

Before	After
-0.8711	-0.1352
0.8488	-0.0675

Before	After
0.2008	-0.0928
0.1809	-0.0949

Q3: Planar Homography from Point Correspondences (30 points)

Algorithm Description:

Step 1: Selecting Point Correspondences

• The user is prompted to select four corresponding points on the background image using mouse clicks. These points should be selected in a specific order (e.g., clockwise or counterclockwise) to match the corners of the overlay image.
• The selected points are visually indicated on the background image with circles and indices.

Step 2: Constructing the Homography Matrix

• Given the source points (corners of the overlay image) and the destination points (user-selected points on the background image), the homography matrix ( H ) is computed.
• The matrix is constructed by solving the linear system formed by the correspondences using Singular Value Decomposition (SVD). The resulting matrix ( H ) transforms the overlay image such that its corners match the selected points in the background image.

Step 3: Overlaying the Images

• The overlay image is warped using the computed homography matrix and then combined with the background image. A mask is used to ensure that the warped overlay image is correctly placed on the background image without blending errors.
• The final image is saved as q3_result.png, displaying the overlay image correctly aligned with the background image.

Usage

1. Run the script with the following command in q3.py file:

    python q3.py desk-normal.png desk-perspective.png 

2. Select Four Points on the background image in clockwise order. These points should correspond to the corners of the overlay image.

3. The script will compute the homography matrix and display the final overlaid image.

4. The following output images will be saved:

   • q3_annotated.png: Background image with selected points annotated.
   • q3_result.png: Final image with the overlay correctly aligned on the background.

Q4: Bonus: Metric Rectification from Perpendicular Lines (10 points)

Cosine similarities result is shown below:

Pair	Before Rectification	After Rectification
1	0.1542	0.0863
2	0.1551	0.1431
3	0.0026	-0.0247
4	-0.1664	0.0713
5	-0.1767	-0.1169

My image:

Pair	Before Rectification	After Rectification
1	-0.1408	0.0246
2	-0.1535	0.0305
3	-0.1959	0.0146
4	0.1638	-0.0148
5	0.1526	-0.0133

Usage

1. Run the script with the following command in q4.py file:

    python q4.py tiles5.JPG 

2. Annotate at least 5 pairs of perpendicular lines (10 lines in total) on the original image for metric rectification.

Q5: Bonus: More Planar Homography from Point Correspondences (10 points)

Above is background.png, desk-normal.png, image2.png, image3.png, image4.png

Result:

Algorithm is similar with q3, this bonus focuses on how to overlay >=3 images on top of some images.

Usage

1. Run the script with the following command in q5.py file:

    python q5.py desk-normal.png image2.png image3.png image4.png background.png 

2. Select Four Points on the background image in clockwise order. These points should correspond to the corners of the overlay image. Then click Enter.

3. Select 2-round four points. Then click Enter.

4. Select 3-round four points. Then click Enter. And so on.

Overall Usage under main.py

If you are under the main.py file, run:

 python main.py --script [q1 | q2 | q3 | q4 | q5] --args [script-specific arguments] 

For example for q1:

 python main.py --script q1 --args tiles5.JPG 

For example for q2:

 python main.py --script q2 --args tiles5.JPG 

For example for q3:

 python main.py --script q3 --args desk-normal.png desk-perspective.png 

For example for q4:

 python main.py --script q4 --args tiles5.JPG 

For example for q5:

 python main.py --script q5 --args desk-normal.png image2.png image3.png image4.png background.png