Assignment 1: Rendering Basics with PyTorch3D

yunxianm

Renders use 128 x 128 pixels per view and 36 evenly spaced views per turntable. The target playback rate is 12 frames per second.

1. Cameras

1.1. Cow Turntable

The cow has uniform vertex color RGB = (0.7, 0.7, 1.0). The camera orbits at distance 3.5 and elevation 20 degrees.

Cow turntable

1.2. Dolly Zoom

starter/dolly_zoom.py varies the field of view from 20 to 80 degrees and back. The cow and ground plane stay fixed. With R = I and T = [0, 0, d]:

d = 3.5 * tan(60 degrees / 2) / tan(fov / 2)

This holds the projected scale at the reference depth constant. Other depths change scale, producing the dolly zoom effect.

Dolly zoom

2. Meshes

2.1. Tetrahedron

The tetrahedron has 4 manually defined vertices and 4 triangular faces. Its vertices are (0.8, 0.8, 0.8), (-0.8, -0.8, 0.8), (-0.8, 0.8, -0.8), and (0.8, -0.8, -0.8). All faces point outward. RGB = (0.16, 0.65, 0.85).

Tetrahedron turntable

2.2. Cube

The cube has 8 manually defined vertices and 12 triangular faces. Coordinates are all combinations of +/-0.8. Each square side uses two outward-facing triangles. RGB = (0.82, 0.40, 0.15).

Cube turntable

3. Re-texturing

The front is blue, color1 = [0, 0, 1], and the back is red, color2 = [1, 0, 0]. Colors interpolate along the original z coordinates:

alpha = (z - z_min) / (z_max - z_min)
color = (1 - alpha) * color1 + alpha * color2

Thus z_min is blue and z_max is red. Vertex colors stay fixed during rendering.

Gradient cow

4. Camera Transformations

The initial camera has R0 = I, T0 = [0, 0, 3], and FOV = 60 degrees. The assignment uses:

R = R_relative @ R0
T = R_relative @ T0 + T_relative

PyTorch3D uses X_view = X_world @ R + T, with +X left and +Y up. The camera center is C = -T @ R.T. Results follow the reference order transform1, transform3, transform4, transform2.

4.1. Clockwise Rotation

R_relative = [[ 0, 1, 0],
              [-1, 0, 0],
              [ 0, 0, 1]]
T_relative = [0, 0, 0]

The image rotates clockwise by 90 degrees. The camera center remains (0, 0, -3), and final T remains [0, 0, 3].

Clockwise rotation

4.2. Moving Back

R_relative = [[1, 0, 0],
              [0, 1, 0],
              [0, 0, 1]]
T_relative = [0, 0, 2]

Final T is [0, 0, 5], so the camera moves to (0, 0, -5). The cow shrinks without changing orientation or FOV.

Camera moved back

4.3. Left and Down

R_relative = [[1, 0, 0],
              [0, 1, 0],
              [0, 0, 1]]
T_relative = [0.5, -0.5, 0]

Final T is [0.5, -0.5, 3], and the camera center is (-0.5, 0.5, -3). Positive view-space X moves the cow left; negative Y moves it down.

Cow shifted left and down

4.4. Side View

R_relative = [[ 0, 0, 1],
              [ 0, 1, 0],
              [-1, 0, 0]]
T_relative = [-3, 0, 3]

The camera looks toward the origin from (-3, 0, 0). Rotation maps T0 to [3, 0, 0]; T_relative corrects this to final T = [0, 0, 3].

Side view

5. 3D Representations

5.1. RGB-D Point Clouds

The supplied unproject_depth_image function converts valid foreground pixels into world-space points and colors. The views contain 62,649 and 102,187 points; their concatenated union contains 164,836.

The columns show view 1, view 2, and the union under identical cameras. The union fills some missing surfaces, but occlusions and depth noise leave gaps. Framing uses the union's median and 95th-percentile radius. All valid points are retained. Camera up = (0, -1, 0) and elevation -12 degrees keep the plant upright.

View 1, view 2, and union

5.2. Parametric Surfaces

Torus

A 160 x 160 grid samples u, v in [0, 2*pi), producing 25,600 points:

x = (R + r * cos(v)) * cos(u)
y = r * sin(v)
z = (R + r * cos(v)) * sin(u)
R = 0.85
r = 0.30

Normalized coordinates determine color. The inner radius is 0.55, and the hole opens along the y axis. Camera elevation is 35 degrees.

Parametric torus

Flower Ring

The additional surface varies the torus radius and centerline height:

R(u) = 0.78 + 0.18 * cos(5*u)
x = (R(u) + 0.16 * cos(v)) * cos(u)
y = 0.10 * sin(3*u) + 0.16 * sin(v)
z = (R(u) + 0.16 * cos(v)) * sin(u)

Flower ring

5.3. Implicit Surfaces

Torus

The torus is the zero level set of:

F(x, y, z) = (sqrt(x*x + z*z) - R)**2 + y*y - r*r
R = 0.85
r = 0.30

Marching cubes extracts the surface from a grid of 80 x 80 x 80 samples over [-1.4, 1.4] on each axis. Grid indices are mapped back to world coordinates.

Implicit torus

Mesh vs. Point Cloud

Property Point cloud Mesh
Geometry 25,600 points 11,200 vertices, 22,400 triangles
Input storage 600.0 KiB 787.5 KiB
Construction 0.0018 s 0.0088 s
Median frame time 8.9637 s 27.6401 s

Frame times use CPU rendering at 128 x 128, one warmup, and three measurements, including CPU image conversion. Shading and rasterization differ between the two pipelines, which limits the comparison.

Point clouds need no connectivity and work directly with RGB-D data. Sparse points leave gaps; larger splats can blur boundaries. Meshes provide continuous surfaces and normals for lighting, but implicit extraction costs time and depends on grid resolution.

Meshes store face indices and share vertices. Point clouds omit faces but may need dense sampling. The table counts input tensors only, excluding renderer buffers and peak memory. The implicit float32 grid adds 2000.0 KiB during construction.

Superellipsoid

The additional implicit surface is a rounded box:

F(x, y, z) = abs(x / 0.85)**4 + abs(y / 0.65)**4
            + abs(z / 0.80)**4 - 1

Marching cubes extracts its zero level set.

Superellipsoid

6. Orbital Garden

The scene has 14 components: a central sphere, three torus orbits, nine satellites, and a pedestal. Each orbit is rotated into a different plane. Its satellites use the same rotation to stay on the orbit.

Orbital Garden

7. Sampling Mesh Surfaces

Faces are sampled with probability proportional to triangle area. Within each selected face, independent uniform values u, v in [0, 1] give:

w0 = 1 - sqrt(u)
w1 = sqrt(u) * (1 - v)
w2 = sqrt(u) * v
p = w0 * vertex0 + w1 * vertex1 + w2 * vertex2

The square root makes sampling uniform over area. The same weights interpolate colors.

Columns show 10, 100, 1,000, and 10,000 points, then the original mesh. Each point set stays fixed; all columns share cameras and scale. More samples cover the cow more densely.

Sampled point clouds and original mesh