16822 Assignment 2¶
by Zi Wang (ziwang2)
The projection matrix is:
$$P=\begin{bmatrix} 6.43169368e+03 & -2.94843744e+03 & 1.14658061e+03 & 2.22724350e+03 \\ -9.34819249e+02 & -6.75486473e+03 & 2.02949013e+03 & 1.82218778e+03 \\ 5.79307220e-01 & -1.42385366e+00 & -7.35268478e-01 & 1.00000000e+00 \end{bmatrix} $$
| Input Image | Surface Points | Bounding Box |
|---|---|---|
 |
 |
 |
(b) Cuboid (15 points)¶
In this problem, we use the same cuboid and annotations as in the assignment instruction. Then we make 0->1, 0->2, 0->3 as x, y and z axes. We then assume that the length of each edge of the cuboid is 1. The projection matrix is:
$$P=\begin{bmatrix} 5.15476162e+01 & -3.40807035e+01 & -7.87617737e-01 & 8.38265585e+01 \\ -1.30709922e+01 & -1.04459670e+01 & -5.54844493e+01 & 9.09992905e+01 \\ 8.23214178e-02 & 1.03468230e-01 & -1.35091411e-02 & 1.00000000e+00 \end{bmatrix} $$
| Input Image | Annotated 2D points | Edges |
|---|---|---|
 |
 |
 |
| Input Image | Annotated Parallel Lines | Vanishing points and principal point |
|---|---|---|
 |
 |
 |
The intrinsic matrix K is: $$K=\begin{bmatrix} 1.15417802e+03 & 0.00000000e+00 & 5.75066005e+02 \\ 0.00000000e+00 & 1.15417802e+03 & 4.31939090e+02 \\ 0.00000000e+00 & 0.00000000e+00 & 1.00000000e+00 \\ \end{bmatrix} $$
Implementation details:
In this question, we need to calculate the Image of the Absolute Conic (IAC) matrix and intrinsic matrix using the vanishing points. There are several key steps to solve this problem.
Vanishing points calculation: The vanishing points are calculated by finding the intersection of pairs of lines using the cross product.
The Image of the Absolute Conic (IAC) matrix $\omega$ is estimated by solving the constraints derived from the vanishing points using Singular Value Decomposition (SVD). Given the vanishing points, the IAC matrix $\omega$ is formulated from the following linear constraints: $$ A = \begin{bmatrix} v_{i_x} \cdot v_{j_x} + v_{i_y} \cdot v_{j_y} & v_{i_x} + v_{j_x} & v_{i_y} + v_{j_y} & 1 \end{bmatrix} $$ Where $v_i$ and $v_j$ are pairs of vanishing points.
The final IAC matrix $ \omega $ is derived as: $$ \omega = \begin{bmatrix} w_1 & 0 & w_2 \\ 0 & w_1 & w_3 \\ w_2 & w_3 & w_4 \end{bmatrix} $$
- Intrinsic Matrix Calculation:The intrinsic matrix $K$ is computed by applying Cholesky decomposition to $\omega$ and normalizing: $$ \omega=K^{-T} K^{-1}, K = L^{-T} $$ Where $L$ is the lower triangular matrix from the Cholesky decomposition of $\omega$.
(b) Camera calibration from metric planes¶
| Input Image | Annotated Square 1 | Annotated Square 2 | Annotated Square 3 |
|---|---|---|---|
 |
 |
 |
 |
| Angle between planes(degree) | |
|---|---|
| Plane 1 & Plane 2 | 67.28242526445507 |
| Plane 1 & Plane 3 | 92.2034855885843 |
| Plane 2 & Plane 3 | 94.71069301661772 |
The intrinsic matrix is: $$K=\begin{bmatrix} 1.07692614e+03 & -4.52638065e+00 & 5.11568923e+02 \\ 0.00000000e+00 & 1.07626752e+03 & 3.95526278e+02 \\ 0.00000000e+00 & 0.00000000e+00 & 1.00000000e+00 \\ \end{bmatrix} $$
Implementation Details:
We need to compute the intrinsic matrix from 3 annotated planes. The steps followed are:
For each plane, we compute the homography $H_i$ that maps $(0, 1), (1, 1), (1, 0), (0, 0)$ to the corners of the plane in the image.
For each $H_i=[h_1, h_2, h_3]$, we have the following constraints:
\begin{equation*} h^T_1 \omega h_2 = 0 \end{equation*}
\begin{equation*} h^T_1 \omega h_1 = h^T_2 \omega h_2 \end{equation*}
- Thus, we can formulate 2 constraints for each $H_i$:
$$ A_i=\begin{bmatrix} h_{1,0} \cdot h_{2,0} & h_{1,0} \cdot h_{2,1} + h_{1,1} \cdot h_{2,0} & h_{1,0} \cdot h_{2,2} + h_{1,2} \cdot h_{2,0} & h_{1,1} \cdot h_{2,1} & h_{1,1} \cdot h_{2,2} + h_{1,2} \cdot h_{2,1} & h_{1,2} \cdot h_{2,2} \\ h_{1,0}^2 - h_{2,0}^2 & 2 \cdot \left( h_{1,0} \cdot h_{1,1} - h_{2,0} \cdot h_{2,1} \right) & 2 \cdot \left( h_{1,0} \cdot h_{1,2} - h_{2,0} \cdot h_{2,2} \right) & h_{1,1}^2 - h_{2,1}^2 & 2 \cdot \left( h_{1,1} \cdot h_{1,2} - h_{2,1} \cdot h_{2,2} \right) & h_{1,2}^2 - h_{2,2}^2 \\ \end{bmatrix} $$
- Solving $Ax=0$ by SVD, we can get the IAC $\omega$. Using similar steps in the previous question, we can decompose $\omega$ and get the intrinsic matrix $K$.
Q3: Single View Reconstruction¶
| Input Image | Annotations | Reconstruction View 1 | Reconstruction View 2 |
|---|---|---|---|
 |
 |
 |
 |
The intrinsic matrix is: $$K=\begin{bmatrix} 814.77013455 & 0. & 516.18611901 \\ 0. & 814.77013455 & 357.12926757 \\ 0. & 0. & 1. \\ \end{bmatrix} $$
Implementation Details:
- Use the algorithm in Q2a to compute the intrinsic matrix $K$.
- Compute plane normals by using $n=K^{-1}v$, where $v$ is the vanishing point.
- Pick one point $P_0$ on the plane as reference and set its depth (in my implementation, depth=1 for the first time).
- Compute plane equation given the known 3D point. The equation could be formulated as: $$ax+by+cz+d=0,$$ where $(a, b, c)$ is the normal vector, $d=-n \cdot P_0$.
- Compute 3D coordinate of all points on the plane via ray-plane intersection. Assuming the direction of a ray corrsponding to a pixel $(x, y)$ in the image would be $r=K^{-1}[x~y~1]^{T}$ and the corresponding 3D point would be $P$, then we have: $$P=r \frac{-d}{n\cdot r}$$
- Repeat the above steps to obtain equations for all planes (and all 3D points). This process will require us to select reference points from previous planes (like the point on the intersection of two planes).