Geometry-based Methods in Vision

Assignment 2

Q1a:

• Matrix P:
[[ 6.43169368e+03 -2.94843744e+03 1.14658061e+03 2.22724350e+03]
[-9.34819249e+02 -6.75486473e+03 2.02949013e+03 1.82218778e+03]
[ 5.79307220e-01 -1.42385366e+00 -7.35268478e-01 1.00000000e+00]]
• original:

• surface points:

• bounding box:

• Test Lines on Affine-Rectified Image:

Q1b:

• Matrix P:
[[ 4.45518881e+01 1.30626594e+02 -3.94871611e+00 2.32450118e+02]
[ 2.61980417e+01 -1.20628026e+01 -1.34706382e+02 5.94993184e+02]
[-2.61820137e-02 1.13867737e-02 -8.73205966e-03 1.00000000e+00]]
• original:

• result:

Q2a:

• K:
[[1.15417802e+03 0.00000000e+00 5.75066005e+02]
[0.00000000e+00 1.15417802e+03 4.31939090e+02]
[0.00000000e+00 0.00000000e+00 1.00000000e+00]]
• original:

• annotations:

• vanishing points and principal point:

Description:

1.1 Vanishing Point Calculation

1. Extract Annotated Parallel Lines: Read the coordinates of annotated parallel lines provided in the file data/q2/q2a.npy. Each pair of parallel lines corresponds to one vanishing point.

2. Calculate Vanishing Points: Vanishing points are determined by calculating the intersection of two lines. For each pair of parallel lines l1 and l2, the intersection (vanishing point) can be found using the following formula:

where l1 and l2 are represented as line equations, and × denotes the cross product.

1.2 Estimation of the Camera Intrinsic Matrix

1. Principal Point Calculation: Given three orthogonal vanishing points vx, vy, vz, their relationship with the camera intrinsic matrix K is defined as follows:

Using these three orthogonality constraints, the camera intrinsic matrix K can be derived.

2. Solve for K: Under the assumption of zero skew and square pixels, the intrinsic matrix K has the following form:

where f is the focal length, and (cx, cy) are the principal point coordinates.

1.3 Implementation and Output

• The function get_K() is used to estimate the camera intrinsic matrix K based on the vanishing points.
• The output includes a visualized image with marked vanishing points and the numerical values of the intrinsic matrix.

2. Derivation of Relevant Equations

1. Line Equations and Vanishing Point:

Given two line equations l1 = (a1, b1, c1) and l2 = (a2, b2, c2), the intersection (vanishing point) can be computed using the cross product:

2. Orthogonality Relationship Between Vanishing Points:

The orthogonality relationship between vanishing points can be expressed as:

where w = K-T K-1 is a transformation matrix related to the camera intrinsic matrix.

Q2b:

• K:
[[1.10864441e+03 4.55495196e+01 4.76234883e+02]
[0.00000000e+00 1.30315829e+03 6.56629618e+02]
[0.00000000e+00 0.00000000e+00 1.00000000e+00]]
plane 1 and plane 2: 69.19551441300955
plane 1 and plane 3: 101.58883813742617
plane 2 and plane 3: 103.288895253722
• original:

• Annotated square1:

• Annotated square2:

• Annotated square2:

1. Image Annotation and Plane Extraction:

• The input image data/q2b.png is read, and the annotation file data/q2/q2b.npy containing three plane annotations (square regions) is loaded.
• The annotate_planes function is used to load the annotated points and convert them into plane contours. Each plane is represented by its corner points and is visualized for verification.

2. Homography Matrix Computation:

• Using the function get_H, the algorithm computes the homography matrix H for each plane. This matrix describes the geometric transformation between the plane and the image, and it is a crucial element for calculating the camera intrinsics.
• For each homography matrix H, the function get_IAC_constraint generates constraints for the Image of the Absolute Conic (IAC) matrix. The IAC matrix characterizes the geometric properties of all planes in the image and is used to determine the camera intrinsics.

3. IAC Constraint Solving and Intrinsic Matrix Calculation:

• The algorithm performs Singular Value Decomposition (SVD) on the combined constraint matrix from all planes to solve for the IAC matrix. The IAC matrix is reshaped into a 3 x 3 form, which encodes the geometric constraints for the camera.
• The function validate_H_constraints is used to verify if the homography matrices satisfy the constraints imposed by the IAC matrix, ensuring that the obtained IAC is correct.
• Finally, using the function calculate_intrinsic_matrix, the intrinsic matrix K is calculated from the IAC matrix using Cholesky Decomposition. This matrix is normalized so that the bottom-right element is 1.

4. Calculating Angles Between Planes:

• The corner points of each plane are converted into homogeneous coordinates. Using the function compute_plane_angles, the algorithm calculates the angles between each pair of planes, using the intrinsic matrix K.

5. Final Output:

The final output includes the calculated camera intrinsic matrix K and the computed angles between each pair of planes to verify the accuracy of the calibration.

Relevant Equations

Q3a:

• K:
[[808.1980502 0 500.56615413]
[ 0 808.1980502 358.67723635]
[ 0 0 1 ]]
• inpput:

• annotations:

• reconstructed view1:

• reconstructed view2:

Description:

1. Camera Projection Model and Intrinsic Matrix

In single view geometry, all 2D image coordinates are obtained through the projection of 3D spatial points. The projection model is usually expressed in the homogeneous coordinate form:

s · x = K · [R | t] · X

where:

• X = (X, Y, Z, 1)T is the homogeneous coordinate of a 3D point in space.
• x = (x, y, 1)T is the homogeneous coordinate of a point in the image.
• s is a scaling factor, used to represent the proportional relationship in homogeneous coordinates.
• K is the camera intrinsic matrix, which is typically of the form:

          K = | fx  0   cx |
              | 0    fy  cy |
              | 0     0     1 |
        

where fx, fy are the focal lengths in the horizontal and vertical directions, and cx, cy are the coordinates of the principal point (the intersection of the optical axis and image plane).

2. Vanishing Points and Linear Constraints

Parallel lines in the real world will converge at a point in the image, called a vanishing point. The vanishing point is computed based on the intersection of line equations. Given two line equations:

l1 = (a1, b1, c1)    and    l2 = (a2, b2, c2)

The intersection can be computed using:

v = l1 × l2

where × denotes the cross product.

3. Ray-Plane Intersection

For each pixel in the image, the corresponding spatial ray direction can be derived using the inverse of the intrinsic matrix:

r = K-1 · x

where r is the direction vector of the ray, and x is the pixel coordinate in the image plane.

4. Plane Equation and Normal Vector Calculation

If multiple points on a plane in the image are known, the plane equation can be derived. Given three non-collinear points (P1, P2, P3), the normal vector n can be calculated using the cross product:

n = (P2 − P1) × (P3 − P1)

5. Color Mapping and 3D Point Cloud Generation

Each 3D point can be assigned a color value based on its corresponding 2D pixel in the image. For a point in the image at (x, y), the color value C is given by:

C = I(y, x)

6. View Transformation and 3D Visualization

Once the 3D point cloud is generated, different view transformations can be applied to visualize the 3D structure from various angles. A view transformation typically uses a rotation matrix or a view transformation matrix to change the observation perspective:

          V = | R  t |
              | 0  1 |
        

where R is a rotation matrix and t is a translation vector. By applying V to the 3D point cloud, different views of the 3D image can be obtained.