16-825 Assignment 1: Rendering Basics with PyTorch3D

Learning for 3D Vision · Abhikhya Tripathy (abhikhyt)

1. Practicing with Cameras

1.1 360-degree Renders

The camera orbits the cow at distance 3 with 36 equally spaced azimuths (look_at_view_transform). The point light moves with the camera so the visible side is always lit.

360-degree render of the cow
360-degree turntable of the cow mesh.

1.2 Re-creating the Dolly Zoom

The field of view sweeps from 5° to 120°. To keep the cow the same size, the width visible at the cow's depth, w = 2·d·tan(fov/2), is held constant (w = 3), so the camera distance for each frame is d = w / (2·tan(fov/2)).

Dolly zoom
Dolly zoom: the cow stays the same size while the background warps.

2. Practicing with Meshes

2.1 Constructing a Tetrahedron

Vertices at (0,0,0), (1,0,0), (0,1,0), (0,0,1) (centered at the origin), with one triangle for each set of three vertices, ordered so every face normal points outward. Single-color texture.

Vertices: 4  |  Triangle faces: 4

360-degree render of a tetrahedron
Single-color texture.
360-degree render of a tetrahedron with colored vertices
One color per vertex, which makes the faces easier to tell apart.

2.2 Constructing a Cube

The 8 corners of a unit cube centered at the origin. Each of the 6 square sides is split into 2 triangles, wound counter-clockwise as seen from outside the cube. Single-color texture.

Vertices: 8  |  Triangle faces: 12

360-degree render of a cube
Single-color texture.
360-degree render of a cube with colored vertices
Corners colored by position, which makes the sides easier to tell apart.

3. Re-texturing a Mesh

color1 = [0, 0, 1] (blue) for the front of the cow (smallest z) and color2 = [1, 0, 0] (red) for the back (largest z). Each vertex is colored by alpha·color2 + (1 − alpha)·color1, with alpha = (z − z_min) / (z_max − z_min).

Retextured cow
Cow with a blue-to-red gradient from front to back.

4. Camera Transformations

The new extrinsics are R = R_relative @ R_0 and T = R_relative @ T_0 + T_relative, with R_0 = I and T_0 = [0, 0, 3].

RenderR_relativeT_relativeDescription
Transform 1 [[0,1,0],[-1,0,0],[0,0,1]] [0,0,0] Rolls the camera 90° about its viewing (z) axis, so the cow appears rotated 90° clockwise in the image. No translation.
Transform 2 Identity [0,0,2] No rotation. Moves the camera 2 units further back along its viewing axis , so the cow appears smaller.
Transform 3 Identity [0.5,-0.5,0] No rotation. Moves the camera 0.5 units right and 0.5 units up, so the cow shifts left and down in the image.
Transform 4 [[0,0,1],[0,1,0],[-1,0,0]] [-3,0,3] Rotates the camera 90° about the vertical (y) axis. Because R_relative @ T_0 alone only turns the camera in place, T_relative moves it to world x = −3, so it looks at the cow from the side.

5. Rendering Generic 3D Representations

5.1 Rendering Point Clouds from RGB-D Images

Each RGB-D image is unprojected to a point cloud with unproject_depth_image. The union is the concatenation of both clouds. Cameras are placed 6 units from the origin with equally spaced azimuths.

Plant point clouds
Left: point cloud from image 1. Middle: point cloud from image 2. Right: union of both.

5.2 Parametric Functions

Torus with ring radius R = 1 and tube radius r = 0.4: x = (R + r·cosθ)·cosφ, y = r·sinθ, z = (R + r·cosθ)·sinφ, sampled on a 100×100 grid.

Torus point cloud
Torus point cloud.
Open can point cloud
New object: an open-top can (a cylinder wall plus a bottom disc).

Open can: side wall (0.6·cosφ, h, 0.6·sinφ) for h ∈ [−0.8, 0.8], and bottom disc (ρ·cosφ, −0.8, ρ·sinφ) for ρ ∈ [0, 0.6]. There is no top, so the can is open.

5.3 Implicit Surfaces

Torus implicit function: F(x,y,z) = (√(x² + z²) − R)² + y² − r² with R = 1, r = 0.4, evaluated on a 64³ voxel grid. The mesh is the zero level-set extracted with marching cubes.

Torus mesh
Torus mesh.
Heart mesh
New object: heart surface.

Heart surface (Taubin): (x² + 9/4·z² + y² − 1)³ − x²y³ − 9/80·z²y³ = 0.

Meshes vs. point clouds

6. (Extra Credit) Do Something Fun

A Glambot-style shot of the cow (as seen on red carpets): a fast camera swoop around the cow on its plane that slows into a close-up of its face. Over 60 frames, distance goes from 6 to 1.6, elevation from 5° to 12°, azimuth sweeps 120°, field of view narrows from 70° to 45°, and the aim point moves from the origin to the cow's face. An ease-out curve s = 1 − (1 − t)³ makes the start fast and the end slow motion, and the shot holds briefly on the final close-up.

Glambot-style shot of the cow
Glambot-style swoop into a close-up of the cow.

7. (Extra Credit) Sampling Points on Meshes

Points are sampled uniformly on the cow's surface:

  1. Choose a face with probability proportional to its area (0.5·|(v1 − v0) × (v2 − v0)|), using torch.multinomial.
  2. Sample uniform barycentric coordinates (1 − √u₁, √u₁(1 − u₂), √u₁·u₂) with u₁, u₂ ~ U(0, 1).
  3. Compute the point as the barycentric combination of the face's three vertices.
Point clouds sampled from the cow mesh
Left to right: 10, 100, 1000 and 10000 sampled points, and the original cow mesh.