HW2: Single-view Reconstruction

Andrew ID: pengliaj

Q1: Camera matrix P from 2D-3D correspondences

(a) Stanford Bunny (15 points)

Algorithm Overview

1. Construct Matrix:
   • Use the Direct Linear Transformation method to construct matrix A such that Ah=0, where h is the vectorized form of the camera matrix H.
2. Solve for H:
   • Perform SVD on A and extract the vector corresponding to the smallest singular value.
   • Reshape this vector into the camera matrix H.
3. Project 3D Points:
   • Use H to project the surface points and bounding box edges onto the image plane.
4. Visualization:
   • Plot the original image with annotated 2D points.
   • Overlay the projected surface points and bounding box on the image.

Camera Matrix ( P )

[[ 6.43169368e+03 -2.94843744e+03  1.14658061e+03  2.22724350e+03]
 [-9.34819249e+02 -6.75486473e+03  2.02949013e+03  1.82218778e+03]
 [ 5.79307220e-01 -1.42385366e+00 -7.35268478e-01  1.00000000e+00]]

Results

Original Image	Annotation

Surface Points	Bounding Box

(b) Cuboid (15 points)

Algorithm Overview

1. Define 3D Coordinates: Assign 3D coordinates to key points on the cuboid, measuring relative dimensions.
2. Annotate Correspondences: Annotate the corresponding 2D points on the image.
3. Compute Camera Matrix: Use the DLT method to compute the camera matrix.
4. Project Cuboid Edges: Define the cuboid's vertices and edges, project them using the camera matrix, and overlay the edges on the image.

Camera Matrix ( P )

[[ 1.95896947e+01  4.04459291e+01 -3.55697156e+00  1.00280898e+02]
 [ 5.02779957e+00 -5.74087050e+00 -4.50659562e+01  2.25027060e+02]
 [-4.74880270e-02  2.11760059e-02 -1.47529934e-02  1.00000000e+00]]

Results

Image	Annotated Image	Result

Q2: Camera calibration K from annotations

(a) Camera calibration from vanishing points (20 points)

Results

Input Image	Annotated Parallel Lines	Vanishing Points and Principal Point

Intrinsic Matrix K

[[1.15417802e+03 0.00000000e+00 5.75066005e+02]
 [0.00000000e+00 1.15417802e+03 4.31939090e+02]
 [0.00000000e+00 0.00000000e+00 1.00000000e+00]]

Algorithm Overview

1. Compute Vanishing Points: For each pair of parallel lines in the image, compute their intersection point, which is the vanishing point.
2. Set Up Equations: Formulate equations based on the orthogonality of directions corresponding to these vanishing points.
3. Solve for ω: Solve the linear system to find the image of the absolute conic ω.
4. Compute K: Extract the intrinsic camera parameters from ω to compute the camera intrinsic matrix K.

Key Equations:

For orthogonal vanishing points, the following relationship applies:

Assuming zero skew and square pixels, the parameterization is given by:

(b) Camera calibration from metric planes (20 points)

Results

Visualization

Input Image	Annotated Square 1	Annotated Square 2	Annotated Square 3

Angles Between Planes

Intrinsic Matrix K

[[1.07229277e+03 4.23052634e+01 4.78652395e+02]
 [0.00000000e+00 1.24563703e+03 7.00149014e+02]
 [0.00000000e+00 0.00000000e+00 1.00000000e+00]]

Algorithm Overview

1. Compute Homographies: Determine the homography for each square mapping to the image plane.
2. Set Up Equations: Use these homographies to formulate equations involving the intrinsic parameters.
3. Solve for B: Solve the system to find the symmetric matrix B = K^{-T} K^{-1}.
4. Compute K: Extract intrinsic parameters from B to compute the camera matrix K.
5. Validate Calibration: Calculate angles between each pair of planes to assess calibration accuracy.

Key Equation:

For each homography H:

where δij = 1 if i = j, and 0 otherwise.

Q3: Single View Reconstruction

(a) (30 points)

Results

Input Image	Annotations	Reconstruction View 1	Reconstruction View 2

Algorithm Overview

1. Compute Intrinsic Matrix: Estimate the matrix by calculating the focal length from vanishing points, assuming the principal point is at the image center.
2. Calculate Plane Normals: For each plane, compute the normal and equation using vanishing points and a reference point.
3. Compute 3D Coordinates: For each pixel on a plane, intersect the ray from the camera center with the plane to obtain 3D coordinates.
4. Visualize Reconstruction: Plot the 3D point cloud from multiple viewpoints.

Relevant Equations:

Intrinsic Matrix K:

Focal Length Estimation:

Ray-Plane Intersection:

where λ = -\frac{d}{\mathbf{n}^\top K^{-1} \tilde{\mathbf{x}}}, d is the plane offset, and n is the plane normal.

How to Run My Code

• Q1a: python q1.py
• Q1b: python q1.py
• Q2a: python q2.py
• Q2b: python q2.py
• Q3: python q3.py