CMU 16-825 Learning For 3D Vision

Assignment 1:
Rendering basics with PyTorch3D

Sichen Guo

Task1 /

Practicing with Cameras

1.1   360-degree Renders

A 360-degree gif of cow mesh.

1.2   Re-creating the Dolly Zoom

A gif with dolly zoom effect.
Task2 /

Practicing with Meshes

2.1   Constructing a Tetrahedron

A green Tetrahedron.

This mesh should have 4 vertices and 4 faces.

2.2   Constructing a Cube

A blue cube.

This mesh should have 8 vertices and 12 faces.

Task3 /

Re-texturing a mesh

3   Re-texturing the cow mesh

Re-textured cow mesh.

In this mesh, color_1 is blue ([0.0, 0.0, 1.0]), and color_2 is red ([1.0, 0.0, 0.0]).

Task4 /

Camera Transformations

4   Original picture

The original picture before camera transformations.

4.1   Transformation 1

R_relative should rotate the original image by 90 degrees around the Z-axis. \[ R_{\mathrm{relative}} = \begin{bmatrix} 0 & 1 & 0 \\ -1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} \qquad T_{\mathrm{relative}} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix} \]

4.2   Transformation 2

R_relative should be the identity matrix, and T_relative should move the camera 2 units far away from the original position along the Z-axis. \[ R_{\mathrm{relative}} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \qquad T_{\mathrm{relative}} = \begin{bmatrix} 0 \\ 0 \\ 2 \end{bmatrix} \]

4.3   Transformation 3

R_relative should be the identity matrix, and T_relative should move the camera 0.5 unit right and 0.5 unit up from the original position. \[ R_{\mathrm{relative}} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \qquad T_{\mathrm{relative}} = \begin{bmatrix} 0.5 \\ -0.5 \\ 0 \end{bmatrix} \]

4.4   Transformation 4

R_relative should rotate the original image by -90 degrees around the Y-axis, and T_relative should move the camera 3 units along the X-axis to the new original position and 3 units far away from the original position along the Z-axis. \[ R_{\mathrm{relative}} = \begin{bmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ -1 & 0 & 0 \end{bmatrix} \qquad T_{\mathrm{relative}} = \begin{bmatrix} -3 \\ 0 \\ 3 \end{bmatrix} \]

Task5 /

Rendering Generic 3D Representations

5.1   Rendering Point Clouds from RGB-D Images

left: rendered from the first image;
middle: rendered from the second image
right: rendered from the union of the first 2 point clouds;

5.2   Parametric Functions

A torus rendered using parametric functions.
A Möbius strip rendered using parametric functions.

5.3   Implicit Surfaces

A torus rendered using implicit surfaces.
A heart rendered using implicit surfaces.

Discussion:
Mesh: Mesh rendering provides continuous surfaces, smooth edges, and better support to render lighting, shadows, and textures. However, it requires correct vertex connectivity, thus rendering speed can be slower, also making data collecting and processing more difficult and increasing memory usage.

Point Cloud: Point cloud rendering is easier to generate from depth cameras or LiDAR and usually requires less structural information and memory usage. Rendering speed can be faster than mesh. However, it may contain noise, holes, and discontinuities, requiring more points and normal information to achieve good visual quality.

Task6 /

Do Something Fun

6.1   Rendering a drum set using mesh

A drum set rendered using mesh.Source: mesh source

The texture was incorrect at first because load_objs_as_meshes function assumes the entire OBJ model uses a single texture image. However, the drum model contains multiple materials: some use KettleDrum.jpg, while others use diffuse colors defined in the MTL file.

I solved this by using load_obj with create_texture_atlas=True, which preserves material assignments of each face and applies the correct textures or material colors.

Use Dolly zoom effect on the drum set.
Task7 /

Sampling Points on Meshes

7   Comparison between different sampling numbers of points

10 points
100 points
1,000 points
10,000 points
100,000 points
Original mesh