**Geometry-based Methods in Vision** **Assignment 1** - Projective Geometry and Homography Affine Rectification ============================================================== Input and output - given images -------------------------------------------------------------- | Input Image | Annotated parallel lines | Affine-Rectified Image | |----------|---------|-----------| | | | | | Test lines on Input Image | Test lines on Affine-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.9868 | 0.9999| |0.9987 | 0.9999| [ Cosine of angle between test lines] | Input Image | Annotated parallel lines | Affine-Rectified Image | |----------|---------|-----------| | | | | | Test lines on Input Image | Test lines on Affine-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.9928 | 0.9999| |0.9958 | 0.9995| [ Cosine of angle between test lines] | Input Image | Annotated parallel lines | Affine-Rectified Image | |----------|---------|-----------| | | | | | Test lines on Input Image | Test lines on Affine-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.7842 | 0.9999| |0.9999 | 0.9999| [ Cosine of angle between test lines] Input and output - custom images -------------------------------------------------------------- | Input Image | Annotated parallel lines | Affine-Rectified Image | |----------|---------|-----------| | | | | | Test lines on Input Image | Test lines on Affine-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |-0.9614 | -0.9999| |0.9992 | 0.9998| [ Cosine of angle between test lines] | Input Image | Annotated parallel lines | Affine-Rectified Image | |----------|---------|-----------| | | | | | Test lines on Input Image | Test lines on Affine-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.9979 | -0.9998| |0.9932 | -0.9992| [ Cosine of angle between test lines] Implementation details --------------------------------------------------------------  As seen above, the goal is to make parallel lines parallel again in affine rectification. We make use of 2 pairs of parallel lines annotated on the transformed image. Using the points of those lines, we take the cross product to get the lines and then take the cross product of the lines to get the point at infinity at which they meet. We get two such points from the two pairs of parallel lines. We take the cross product of these two points to get the line equation of the line at infinity. Using the coefficients of this lines, the homography matrix H can be computed as seen above. This H is then applied on the transformed image to get the rectified image. Metric Rectification ============================================================== Input and output - given images -------------------------------------------------------------- | Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image | |----------|---------|-----------|-----------| | | | | | | Test lines on Input Image | Test lines on Metric-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.1677 | 0.1019| |0.0349 | -0.0537| [ Cosine of angle between test lines] | Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image | |----------|---------|-----------|-----------| | | | | | | Test lines on Input Image | Test lines on Metric-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |-0.2526 | 0.0349| |0.0881 | 0.1032| [ Cosine of angle between test lines] | Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image | |----------|---------|-----------|-----------| | | | | | | Test lines on Input Image | Test lines on Metric-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.6685 | -0.0211| |-0.04480 | -0.0096| [ Cosine of angle between test lines] Input and output - custom images -------------------------------------------------------------- | Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image | |----------|---------|-----------|-----------| | | | | | | Test lines on Input Image | Test lines on Metric-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.1810 | 0.0232| |0.3273 | 0.1133| [ Cosine of angle between test lines] | Input Image | Annotated perpendicular lines on input image | Annotated perpendicular lines on Affine-Rectified Image | Rectified Image | |----------|---------|-----------|-----------| | | | | | | Test lines on Input Image | Test lines on Metric-Rectified Image | |----------|---------| | | | | Before | After | |---|---| |0.4429 | 0.3016| |-0.111 | -0.0888| [ Cosine of angle between test lines] Implementation details -------------------------------------------------------------- Here we perform affine rectification first as described above and then use 2 pairs of perpendicular lines to perform metric rectification. The overall algorithm is as shown below.  From the 2 pairs of perpendicular lines, we find C_infintiy_star_dash using the below equations.   Since affine rectification is already performed, d,e,f are 0.  After taking SVD on C_infintiy_star_dash, we make use of the singular values and left singular vector to compute the homography matrix H as shown above. This H is then applied on the affine-rectified image to get the metric-rectified image. Note: The above metric rectification can be also done without performing affine rectification using parallel lines as a first step. The same method as described above can be used except that d, e and f won't be 0 in C_infintiy_star_dash. We would need atleast 5 pairs of perpendicular lines to do this. In the time that I had, I couldn't get very accurate results for this, hence they haven't been included here. My code for this can be reviewed in canvas submission. Planar Homography from Point Correspondences ============================================================== Input and output - given images -------------------------------------------------------------- | Normal Image | Perspective Image | Annotated corners in Perspective Image | Warped and Overlaid Image | |----------|---------|-----------|-----------| | | | | | Input and output - custom images -------------------------------------------------------------- | Normal Image | Perspective Image | Annotated corners in Perspective Image | Warped and Overlaid Image | |----------|---------|-----------|-----------| | | | | | Implementation details -------------------------------------------------------------- We make use of 4 point correspondences between the normal image and perspective image and solve a linear system of equations.  We construct A matrix as shown above using the points and take SVD of A. We get the homography matrix H from the last row of the right singular vector after SVD and we can apply this homography on the normal image to get a warped image. This warped image is then blended onto the perspective image. More Planar Homography from Point Correspondences ============================================================== Heroes of Deep Learning on Times Square Billboards  | LeCun | Karpathy | Hinton | Bengio | Goodfellow | |----------|---------|-----------|-----------|-----------| | | | | |  Implementation is the same as what is done in Planar Homography. This is just repeated planar homography for all the 5 images.