16-822 Geometry-based Methods in Vision

Assignment 2: Single-view Reconstruction

Qitao Zhao (qitaoz), Fall 2024

Q1: Camera matrix P from 2D-3D correspondences

(a) Stanford Bunny

Surface Points	Bounding Box

(b) Cuboid

Input Image	Annotated 2D points	Edges

Q2: Camera calibration K from annotations

(a) Camera calibration from vanishing points

Input Image	Annotated Parallel Lines	Vanishing points and principal point

$\begin{array}{c}K=\left[\begin{array}{ccc}1.15417802e+03& 0.00000000e+00& 5.75066005e+02\\ -0.00000000e+00& 1.15417802e+03& 4.31939090e+02\\ -0.00000000e+00& 0.00000000e+00& 1.00000000e+00\end{array}\right]\end{array}$$K = \begin{bmatrix} 1.15417802e+03 & 0.00000000e+00 & 5.75066005e+02 \\ -0.00000000e+00 & 1.15417802e+03 & 4.31939090e+02 \\ -0.00000000e+00 & 0.00000000e+00 & 1.00000000e+00 \end{bmatrix}$

Brief Description of the Implementation

1. Vanishing Point Calculation

We first compute vanishing points from annotated lines. Each pair of 2D points forms a line, and two such lines are used to compute a vanishing point — the intersection of the two lines. The vanishing points represent directions in 3D space where parallel lines converge when projected onto the image plane.

For each pair of annotated points ( p_1(x_1, y_1) ) and ( p_2(x_2, y_2) ), the line equation is computed using the form:

The intersection of two such lines gives the vanishing point ( (x, y) ).

2. Forming the Equation for IAC

In the case of square pixels, the image of the absolute conic is assumed to have the form:

Each pair of vanishing points ( v_i ) and ( v_j ) generates a constraint in the form of the equation:

Using three pairs of vanishing points, these constraints are stacked to form a system of linear equations in the matrix form:

Where ( A ) is a ( 3 $\times$$\times$ 4 ) matrix, and ( w ) is the vector of unknowns ( [w_1, w_2, w_3, w_4] ).

3. Solving for $\omega$$\omega$

The vector ( w ) is obtained as the null vector of ( A ), which is found by solving the system ( A $\cdot$$\cdot$ w = 0 ) using SVD. The last row of the matrix ( V^T ) from the SVD solution corresponds to the vector ( w ).

This determines the matrix ( $\omega$$\omega$ ).

4. Estimating the Camera Intrinsic Matrix ( K )

Once ( $\omega$$\omega$ ) is determined, the camera intrinsic matrix ( K ) is computed from the relation:

To extract ( K ), Cholesky factorization is applied to ($\omega$$\omega$), and the result is then inverted. Finally, the matrix ( K ) is normalized such that the last element ( K_{33} = 1 ).

(b) Camera calibration from metric planes

Input Image	Annotated Squares

$\begin{array}{c}K=\left[\begin{array}{ccc}1.08447642e+03& -1.35121131e+01& 5.20013594e+02\\ 1.17407507e-13& 1.07900526e+03& 4.02544642e+02\\ -0.00000000e+00& 0.00000000e+00& 1.00000000e+00\end{array}\right]\end{array}$$K = \begin{bmatrix} 1.08447642e+03 & -1.35121131e+01 & 5.20013594e+02 \\ 1.17407507e-13 & 1.07900526e+03 & 4.02544642e+02 \\ -0.00000000e+00 & 0.00000000e+00 & 1.00000000e+00 \end{bmatrix}$

Brief Description of the Implementation

1. Homography Computation

For each planar square, we compute the homography H that maps the known corner points of the square to their corresponding imaged 2D points. These corner points are:

• (0, 0), (0, 1), (1, 0), (1, 1)

Given the observed image points for each square, the homography H is computed, which describes the transformation between the real-world plane of the square and its image.

2. Imaged Circular Points

Once the homography H for each square is obtained, we compute the imaged circular points. The circular points are defined as (1, $\pm$$\pm$ i, 0), where i is the imaginary unit. Using the homography, these circular points are projected into the image as:

Given that ( H = [h_1, h_2, h_3] ) (where ( h_1 ), ( h_2 ), and ( h_3 ) are the column vectors of the homography matrix), the imaged circular points become:

These points lie on the image of the absolute conic (IAC), which allows us to impose constraints on the camera’s intrinsic matrix.

3. Fitting the Conic ( $\omega$$\omega$ )

The IAC encodes the intrinsic parameters of the camera. The constraint that the imaged circular points ( h_1 \pm i h_2 ) lie on ( $\omega$$\omega$ ) provides two real constraints:

These are linear equations in the elements of ( \omega ), the conic we are solving for. With multiple squares, we generate more than five equations, which is sufficient to solve for ( \omega ) up to a scale factor.

4. Calibration Matrix ( K )

Once the conic ( \omega ) is computed, the final step is to extract the camera’s intrinsic matrix ( K ) (as what we did in (a)).

Q3: Single View Reconstruction

Input Image	Annotations	Reconstruction View 1	Reconstruction View 2

Brief Description of the Implementation

1. Compute ( K ) from 3 vanishing points: Estimate the camera intrinsic matrix ( K ) using three vanishing points in the image.

2. Select a reference point: Choose a reference point in the image to start the unprojection (we choose two points lying on the same vertical line covering all planes and set their depth as 1, so that we do not need to worry about scale).

3. Unproject the reference point: Convert the reference point from 2D to 3D using the inverse of the intrinsic matrix ( K ) and assign a depth (scale) to the point:

   $$\begin{array}{}\text{(10)}& X_r=K^{-1}x\end{array}$$

4. Find the plane normal and scalar ( a ): Compute the plane's normal vector ( n ) and the scalar ( a ) using with the known 3D point:

   $$\begin{array}{}\text{(11)}& a=-n^T{X}_r\end{array}$$

5. Unproject other points to 3D: Apply the same unprojection method to the remaining 2D points:

   $$\begin{array}{}\text{(12)}& X=K^{-1}x\end{array}$$

6. Repeat for all planes: Repeat the process for every plane in the scene to obtain the 3D geometry.