| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
Zi Wang (ziwang2)
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
|
|
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
For the fundamental matrix, we use 8-point algorithm. First, the points are normalized with transformation $ T_1 $ and $ T_2 $. Here $$\mathbf{T}_i = \begin{bmatrix}s_i & 0 & - s_i x_{io} \\ 0 & s_i & - s_i y_{io} \\ 0 & 0 & 1 \end{bmatrix}$$ $$(x_{io}, y_{io}) = \frac{1}{\mathbf{N}} \sum{p_i}$$ $$s_i = \frac{\sqrt{2}}{d_{avg}}$$ $$d_{avg} = \frac{1}{\mathbf{N}} \sum{\sqrt{(x_i - x_{io})^2 + (y_i - y_{io})^2}}$$ Then, we construct the matrix $ A $ using the normalized points. Each row of $ A $ is written as: $$ A_i = [x_i x'_i, x_i y'_i, x_i, y_i x'_i, y_i y'_i, y_i, x'_i, y'_i, 1]. $$ Next, singular value decomposition (SVD) is applied to $ A $, and the fundamental matrix $ F $ is derived from the last column of the matrix $ V $. To enforce the rank-2 constraint, SVD is performed again on $ F $, and the smallest singular value is set to zero to obtain the final matrix $ F $. Finally, the fundamental matrix is denormalized by applying $ F = T_2^T F T_1 $.
We can further calculate the essential matrix $E$ as $E=K^T_2 F K_1$, where $K1$ and $K_2$ are the intrinsic matrices.
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
We use 7-point algorithm to calculate the fundamental matrix.
The reconstruction of the matrix $A$ is the same as Q1(a), and we also conduct SVD on this matrix. We can then get two possible solusions $F_1$ and $F_2$ using the last two columns of the matrix $V$. The set of possible fundamental matrices is determined by the values of $ \alpha $ that satisfy $ \det(\alpha\mathbf{F}_1 + (1 - \alpha)\mathbf{F}_2) = 0 $. Solving this cubic equation can result in multiple real roots. For each potential fundamental matrix $ \mathbf{F} $, the total error is computed based on how far the corresponding points are from their respective epipolar lines. The fundamental matrix that produces the smallest error is chosen as the best solution. Then we perform the same transformation as Q1(a) to obtain the fundamental matrix.
RANSAC + 8-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 7-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 8-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 7-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 8-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 7-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 8-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
RANSAC + 7-point algorithm
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
During each iteration, a random selection of the required number of correspondences is made from the available set. Using these correspondences, the fundamental matrix $ \mathbf{F} $ is estimated by applying either the 7-point or 8-point algorithm, as described previously. Once the matrix is computed, the error for each correspondence is calculated. For a given pair of points $ (p_1, p_2) $, the corresponding epipolar lines are determined by calculating $ l_1 = \mathbf{F}_i p_1 $ and $ l_2 = \mathbf{F}_i^T p_2 $. The total error is derived by summing the distance between point $ p_2 $ and line $ l_1 $, and the distance between $ p_1 $ and line $ l_2 $.
Afterward, the number of inliers is determined by comparing the total error to a predefined threshold (I set threshold=3). The fundamental matrix $ \mathbf{F} $ candidate that results in the most inliers is tracked throughout the iterations. Once the iterations are complete (the total number of iterations is 10000), the final fundamental matrix is computed using the inliers identified from the best candidate.
| Viewpoint 1 | Viewpoint 2 | Viewpoint 3 |
|---|---|---|
 |
 |
 |
The algorithm uses the projection matrices $ P_1 $ and $ P_2 $ from both cameras, along with their corresponding matched points. For each set of matched points, we aim to compute the 3D point $ X $. Given the relationships $ P_1 X = x_1 $ and $ P_2 X = x_2 $, we can formulate the problem as solving $ P_1 X \times p_1 = 0 $ and $ P_2 X \times p_2 = 0 $. These two equations can be combined into a matrix form $ A X = 0 $, where the matrix $ A $ is defined as:
$$ A = \begin{bmatrix} p_1[0]~P_1[2,:] - P_1[0,:] \\ p_1[1]~P_1[2,:] - P_1[1,:] \\ p_2[0]~P_2[2,:] - P_2[0,:] \\ p_2[1]~P_2[2,:] - P_2[1,:] \end{bmatrix} $$Next, SVD is applied to the matrix $ A $, and the 3D point $ X $ is derived from the last column of the matrix $ V $.
I collected the images from Mip-NeRF 360 dataset.
Scene 1:
Input images: (311 in total)
 |
 |
 |
 |
Output:
Scene 2:
Input images: (125 in total)
 |
 |
 |
 |
Output:
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
| Viewpoint 1 (Points) | Viewpoint 2 (Epipolar Lines) |
|---|---|
 |
 |
I use a SIFT detector to detect keypoints and compute their descriptors for both images. After that, I use a brute-force matcher to find the closest matches between the two sets of descriptors. To filter out false matches, I apply the ratio test: only matches where the closest match is significantly better than the second closest are kept. So far, I can get the matching keypoints from both images, along with the list of good matches. Finally, I use RANSAC and 8-point algorithm previously mentioned to estimate the fundamental matrix.