16-825 Assignment 1: Rendering Basics with PyTorch3D

1. Practicing with Cameras (15 Points)

1.1 360-degree Renders (5 points)

1.2 Re-creating the Dolly Zoom (10 points)

2. Practicing with Meshes (10 Points)

2.1 Constructing a Tetrahedron (5 points)

Vertex / face countThe tetrahedron has 4 vertices and 4 faces.

2.2 Constructing a Cube (5 points)

Vertex / face countThe cube has $8$ vertices and $6\cdot2=12$ faces ($6$ quads, $12$ triangles)

3. Re-texturing a Mesh (10 Points)

Color choice$\text{color}_1 = [0, 0, 1]$, $\text{color}_2 = [1, 0, 0]$

4. Camera Transformations (10 Points)

Transform 1$R_\text{relative}$ rotates $-90°$ about the $z$-axis, $T_{relative} = [0, 0, 3.0]$ pulls the camera away from the cow by $3$ units.

Transform 2$R_\text{relative}=I_3$ and $T_\text{relative}=[0, 0, 5.0]$ meaning we do not rotate the camera and we pull the camera away from the cow by $2$ units.

Transform 3$R_\text{relative}=I_3$ and $T_\text{relative}=[0.5, -0.5, 3.0]$ meaning we do not rotate the camera and we move the camera along the $X$ and $Y$ axes by $0.5$ units.

Transform 4$R_\text{relative}=\begin{bmatrix} \cos(\frac{\pi}{2}) & 0 & \sin(\frac{\pi}{2}) \\ 0 & 1 & 0 \\ -\sin(\frac{\pi}{2}) & 0 & \cos(\frac{\pi}{2}) \\ \end{bmatrix}$ and $T_\text{relative}=[0, 0, 3]$ meaning we rotate the camera about the $Y$ axis by $\frac{\pi}{2}$ radians.

5. Rendering Generic 3D Representations (45 Points)

5.1 Rendering Point Clouds from RGB-D Images (10 points)

5.2 Parametric Functions (10 + 5 points)

5.3 Implicit Surfaces (15 + 5 points)

Mesh vs. point cloud tradeoffsThe difficulty in rendering both meshes and pointclouds in $\texttt{pytorch3d}$ is quite similar given the library's design. Comparing both representations, it is more efficient to render a mesh surface representation depending on the resolution of the mesh. Here I used $N=500$ point samples for the pointcloud torus while I only need a voxel grid of size $64^3$. Of course, marching cubes has a polynomial time complexity of $\mathcal{O}(N^3)$ but the representation is pre-computed and this is parallelizable. Space complexity of the pointcloud (where a pointcloud is defined as the unordered set of points $\mathcal{P}=\{\mathbf{p}_i\}_{i=0}^N, \mathbf{p}\in\mathbb{R}^3$) is $\mathcal{O}(N)$ while the mesh has a larger space complexity of $\mathcal{O}(|V|\times|\mathcal{F}|)$ (when $V=\{\text{set of points } \mathbf{p}_i\in\mathbb{R}^3\}$ and $\mathcal{F}=\{\text{set of faces}\}$ where each face is a tuple of indices $(i,j,k)$ into $V$). In the worst case, consider $|V|=N$ with $|\mathcal{F}|\approx N$ (quite similar to the cow mesh!) in such a case the size complexity is approximately $N\times N$ as we need to keep track of the vertices and their corresponding faces.
$\text{Note: } |\cdot| \text{ denotes cardinality of the set.}$

6. Do Something Fun (Extra Credit, +10 points)

What you didUsing this model I varied the elevation angle and rotation speed around the car.

7. Sampling Points on Meshes (Extra Credit, +10 points)