Assignment 2

16-822 | Qin Han | qinh@andrew.cmu.edu

Q1. Camera matrix P from 2D-3D correspondences

Q1(a) Result

solution

Q1(a) Algorithm Description

The camera matrix \(P\) is estimate by the PnP (Perspective-n-Point) algorithm, based on the formula \(PX\times x=0\).

First, we create the matrix \(A\) by the 2D-3D correspondences. \[ A = [x]^T_{\times} \begin{bmatrix} X^T & 0 &0 \\ 0 & X^T & 0 \\ 0 & 0 & X^T \end{bmatrix} \]

Then we conduct singular value decomposition on the matrix \(A\), and get the camera matrix \(P\) by the last row of the matrix \(V\).

Q1(b) Result

solution

Q1(b) Algorithm Description

Similar to Q1(a), the camera matrix \(P\) is estimate by the PnP (Perspective-n-Point) algorithm, based on the formula \(PX\times x=0\). And I mannually annotated the 2D-3D correspondences for the cube image.

First, we create the matrix \(A\) by the 2D-3D correspondences. \[ A = [x]^T_{\times} \begin{bmatrix} X^T & 0 &0 \\ 0 & X^T & 0 \\ 0 & 0 & X^T \end{bmatrix} \]

Then we conduct singular value decomposition on the matrix \(A\), and get the camera matrix \(P\) by the last row of the matrix \(V\).

Q2. Camera calibration K from annotations

Q2(a) Result

solution

\[ K = \begin{bmatrix} 1154.2 & 0 & 575 \\ 0 & 1154.2 & 432 \\ 0 & 0 & 1 \end{bmatrix} \]

Q2(a) Algorithm Description

I first calculate the vanishing points by the annotations of parallel lines. Then I calculate the camera calibration matrix \(K\) by the vanishing points.

Since we know the camera has zero skew, and that the pixels are square. The camera calibration matrix \(K\) can be represented as: \[ K = \begin{bmatrix} f & 0 & c_x \\ 0 & f & c_y \\ 0 & 0 & 1 \end{bmatrix} \] where \(f\) is the focal length, and \(c_x\) and \(c_y\) are the principal points. So, the \(\omega\) matrix can be represented as: \[ \omega = \begin{bmatrix} 1 & 0 & b \\ 0 & 1 & c \\ b & c & d \end{bmatrix} \] We then compute the \(\omega\) by solving the equation \(v_i^T \omega v_j = 0\) of the 3 pairs of perpendicular vanishing points. And we can get the camera calibration matrix \(K\) by: \(f = \sqrt{(d-(b^2+c^2))} \), \(c_x = -b \), \(c_y = -c \).

Q2(b) Result

solution

Evaluation results:
Angle between planes(degree)
Plane 1 & Plane 2 67.57512
Plane 1 & Plane 3 92.24721
Plane 2 & Plane 3 94.78379

\[ K = \begin{bmatrix} 1084.4 & -13.5 & 520.0 \\ 0 & 1079.0 & 402.5 \\ 0 & 0 & 1 \end{bmatrix} \]

Q2(b) Algorithm Description

We have three suqare planes in the image, so we can first calculate the Homography from standard coordinates to the plane coordinates. This can help us identify the circular points of the planes. Then we can calculate the \(\omega\) matrix by the circular points of the planes, using the formula \( h_1^T \omega h_2 = 0 \) and \( h_1^T\omega h_1 = h_2^T\omega h_2 \).

After we get the \(\omega\) matrix, we know \( \omega = K^{-T}K^{-1} \). So we can calculate the camera calibration matrix \(K\) by:

first, conduct the cholesky decomposition on the \(\omega\) matrix, and get the matrix \(L\). Then we can get the camera calibration matrix \(K\) by: \( K = L^{-T} \).

Q3. Single View Reconstruction

Q3 Result

Q3 Algorithm Description

1. Use Q2a to compute `K`. 2. Compute plane normals:

1. Compute vanishing points for each plane from the annotated parallel lines. 2. Compute the direction of parallel lines from the vanishing points, by \( d = K^{-1} p \) 3. Compute the normal of the plane by the cross product of the two directions.

3. Pick a reference point that is the first corner point of the first plane. 4. Set the depth of the reference point to 1. 5. Compute the plane equation given the known 3D point, based on the formula \( n^T x + a = 0\). 6. Given the intrinsic matrix \(K\), compute rays for each pixel. 7. Use the rays to compute 3D coordinate of all points on the plane via ray-plane intersection, e.g. \( n^T (t \cdot d) + a = 0 \) to get the distance \( t \). 8. Repeat the above steps to obtain equations for all planes (and all 3D points).

Input Image Annotated Image Reconstruction View 1 Reconstruction View 2