HW2: Single-view Reconstruction

cque

Q1: Camera matrix P from 2D-3D correspondences (30 points)

(a) Stanford Bunny (15 points)

Input Image	Annotated 2D points

Input Image	Annotated 2D points

(b) Cuboid (15 points)

Input Image	Annotated 2D points	Result

Q2: Camera calibration K from annotations (40 points + 10 points bonus)

(a) Camera calibration from vanishing points (20 points)

Submission

1. Output plots of the vanishing points and the principal point. Also include visualizations of the annotations that you used. See the following figures:

   Input Image	Annotated Parallel Lines	Vanishing points and principal point

2. $$K = \begin{bmatrix} 1.15417802 \times 10^{3} & 0.00000000 & 5.75066005 \times 10^{2} \\ 0.00000000 & 1.15417802 \times 10^{3} & 4.31939090 \times 10^{2} \\ 0.00000000 & 0.00000000 & 1.00000000 \end{bmatrix}$$

3. Given that camera has zero skew and the pixels are square, we can use 3 vanishing points that are orthogonal to each other to find w.

   $$w = \begin{bmatrix} w_1 & 0 & w_2 \\ 0 & w_1 & w_3 \\ w_2 & w_3 & w_4 \end{bmatrix}$$
   Use $$v_1^T w v_2 = 0 $$ to stack equations: $$Aw = 0 $$
   $$ w = (K K^T) ^{-1}$$
   use cholesky decomposition and inverse to find K

(b) Camera calibration from metric planes (20 points)

1. Visualizations

Input Image	Annotated Square 1	Annotated Square 2	Annotated Square 3

1. Evaluate angles between each pair of planes. This will reflect the correctness of your calibration result.

1. $$ w = (K K^T) ^{-1}$$
2. $$w = \begin{bmatrix} w_1 & w_2 & w_4 \\ w_2 & w_3 & w_5 \\ w_4 & w_5 & w_6 \end{bmatrix}$$

   To find w, we can use 3 homographies to solve for Aw = 0. Each homography creates 2 constraints

   $$h_1^T w h_2 = 0 $$
   $$ h_1^T w h_1 - h_2^T w h_2 = 0 $$

   where each homography:

   $$ H = [h_1,h_2,h_3] $$

(c) Camera calibration from rectangles with known sizes (10 points bonus)

1. Visualizations

Input Image	Annotated Rectangle 1	Annotated Rectangle 2	Annotated Rectangle 3

1. Evaluate angles between each pair of planes. This will reflect the correctness of your calibration result.

1. $$ w = (K K^T) ^{-1}$$
2. $$w = \begin{bmatrix} w_1 & w_2 & w_4 \\ w_2 & w_3 & w_5 \\ w_4 & w_5 & w_6 \end{bmatrix}$$

   To find w, we can use 3 homographies to solve for Aw = 0. Each homography creates 2 constraints

   $$h_1^T w h_2 = 0 $$
   $$ h_1^T w h_1 - h_2^T w h_2 = 0 $$

   where each homography:

   $$ H = [h_1,h_2,h_3] $$
3. Difference from 2b: instead of projecting from a square to 3 squares, we use known height-width ratios to find the homography from an even rectangle to the projected rectangles in the image

Q3: Single View Reconstruction (30 points + 10 points bonus)

In this question, your goal is to reconstruct a colored point cloud from a single image.

(a) (30 points)

Submissions

1. Output reconstruction from at least two different views.

Input Image	Annotations	Reconstruction View 1	Reconstruction View 2

1. Description
2. Used K from q2a to compute normal rays for each plane
   • $$n = d_1 \times d_2$$
   • $$ d_i = K^{-1} v_i $$
   • used 2 sets of parallel lines to compute 2 vanishing points for each plane
3. Computed directions of the ray for each pixel using
   • $$ d_i = K^{-1} x_i (eq. 1) $$
4. Found plane equation
   • $$0 = n^T X + a$$
   • picked a point on 2d and converted to a ray using (eq. 1)
   • picked an arbitrary depth t to convert 2d point to 3d point
     • $$ray(t) = d*t$$
   • solve for a by substituting 3d point for X and used plane normal for n
5. Computed ray-plane intersection for each 2d point by substituting the ray into the plane equation, finding depth, then solving for the 3d coordinate
$$0 = n^T(d*t) + a$$