HW2: Single view Reconstruction

Kewen Wu

Q1: Camera matrix from 2D-3D correspondences.

  1. Compute P

    2. \[ \begin{bmatrix} 6.12468612 \times 10^{-1} & -2.80769806 \times 10^{-1} & 1.09185025 \times 10^{-1} & 2.12092927 \times 10^{-1} \\ -8.90197009 \times 10^{-2} & -6.43243106 \times 10^{-1} & 1.93261536 \times 10^{-1} & 1.73520830 \times 10^{-1} \\ 5.51654830 \times 10^{-5} & -1.35588807 \times 10^{-4} & -7.00171505 \times 10^{-5} & 9.52266452 \times 10^{-5} \end{bmatrix} \]

  2. Visualization
    3. Surface Points Bounding Box
  3. Compute P of a cuboid

    4. \[ \begin{bmatrix} -3.03160098 \times 10^{-1} & 2.06147503 \times 10^{-1} & -9.81416637 \times 10^{-3} & 6.91384023 \times 10^{-1} \\ 9.48867096 \times 10^{-2} & 8.61430420 \times 10^{-2} & -3.35434316 \times 10^{-1} & 5.08471299 \times 10^{-1} \\ -9.27741010 \times 10^{-5} & -1.61381933 \times 10^{-4} & -1.17016445 \times 10^{-5} & 2.36384286 \times 10^{-3} \end{bmatrix} \]

    Input Image Edges Results

    Q2: Camera calibration

      Camera calibration from vanishing points

      Input Image Annotated Parallel Lines Vanishing points and principal point
    1. Report K

      2. \[ \begin{bmatrix} 1154.18 & 0.00 & 575.07 \\ 0.00 & 1154.18 & 431.94 \\ 0.00 & 0.00 & 1.00 \end{bmatrix} \]

    2. Brief description of your implementation

      3. First of all, calculate 3 vanishing points using the three sets of vanishing lines.

      Next, since each set is orthogonal to each other:

      \[ {v_i}^T \mathbf{K}^{-T} \mathbf{K}^{-1} {v_j} = 0 \quad \text{for} \quad i \neq j \]

      Since we assume zero skew and square pixels, the intrinsic matrix simplifies to: \[ \mathbf{K} = \begin{bmatrix} f & 0 & c_x \\ 0 & f & c_y \\ 0 & 0 & 1 \end{bmatrix} \]

      We can solve the K using Cholesky decomposition.

      Camera calibration from metric planes

      Input Image Annotated Image
    3. Evaluate angles
      4. Angle between planes(degree)
      Plane 1 & Plane 2 67.41
      Plane 1 & Plane 3 92.22
      Plane 2 & Plane 3 94.71
    4. Report K

      5. \[ \begin{bmatrix} 1078.90 & -7.69 & 515.66 \\ 0.00 & 1077.52 & 401.12 \\ 0.00 & 0.00 & 1.00 \end{bmatrix} \]

    5. Brief description of your implementation

      6. First of all, calculate the Homography matrix from world to image plane. Assume the world coordinates are [[0,0,1],[1,0,1],[1,1,1],[0,1,1]]

      Next, for every H, the first and second column h1 and h2:

      \[ {h_1}^T \mathbf{K}^{-T} \mathbf{K}^{-1} {h_2} = 0 \]

      \[ {h_1}^T \mathbf{K}^{-T} \mathbf{K}^{-1} {h_1} = {h_2}^T \mathbf{K}^{-T} \mathbf{K}^{-1} {h_2} \]

      Since we don't assume anything in K, so we have 6 DoF. And we have three planes which every plane has two equation.

      We can solve the K using Cholesky decomposition.

      For calculation angles, for each plane, we can first calculate the 2 vanishing points using 2 sets of parallel lines. Then we can calculate the vanishing line using these two points.

      Then use the result we have proved in PS2: \[ n = K^T l \]

      We can get the normals of the plane.

    Q3: Single View Reconstruction

      Input Image Annotations Reconstruction View 1 Reconstruction View 2
    1. Brief description

      2. After calculating the K, I calculate the normals of each plane: \[ d1 = k^{-1} p1 \] \[ d2 = k^{-1} p2 \] \[ n = d1 \times d2 \]

      Then I use one point for reference, set the depth to be 1. Then calculate and normalize the ray and multiply it using this depth. \[ ray = k^{-1} point \]

      Since for the plane: \[ np+a = 0 \]

      We can calculate \(a\) for plane 1 using this reference plane. Then I calculate the four corners of this plane 1.

      \[ t = \frac{-a}{norm \cdot ray} \]

      For the rest of the 4 planes, there would be at least one corner on the plane 1 lies on rest of the each plane. I could calculate \(a\) for every plane.

      I use fillPoly in cv2 to find out all the pixels on each plane. Calculate the depth using the method mentioned above.