Assignment 1: Rendering Basics with PyTorch3D

16-825 Learning for 3D Vision · Yongyi Xiong (andrewid: yongyix)

1. Practicing with Cameras

1.1 360-degree Renders

360 degree turntable render of the cow mesh
36 frames, dist = 3, elev = 0, azimuth swept from 0 to 350 degrees.

1.2 Dolly Zoom

Keeping the cow the same width in frame requires

distance = (width / 2) / tan(fov / 2) = 1.732 / tan(fov / 2)

where 1.732 = 3 · tan(30°) comes from the starter setup (fov = 60°, distance = 3). I also set znear = 0.01, since the default of 1.0 clips through the cow once the camera closes in at wide fov.

Dolly zoom effect on the cow
60 frames, fov from 5° to 120°, distance from 39.7 to 1.0.

2. Practicing with Meshes

2.1 Constructing a Tetrahedron

4 vertices and 4 triangular faces. Each face uses three of the four vertices, so the face list is the four ways of omitting one.

360 degree render of a tetrahedron
Tetrahedron, rendered at elev = 25° so more than one face is visible.

2.2 Constructing a Cube

8 vertices and 12 triangular faces, since each of the 6 square sides splits into 2 triangles.

360 degree render of a cube
Cube, with faces wound so their normals point outward.

3. Re-texturing a Mesh

I chose color1 = [0, 0, 1] (blue) for the front of the cow and color2 = [1, 0, 0] (red) for the back. Each vertex colour is interpolated by its z coordinate:

alpha = (z - z_min) / (z_max - z_min)    color = alpha · color2 + (1 - alpha) · color1
Cow with a blue to red gradient texture
Blue at the head, red at the rear, purple in between.

4. Camera Transformations

Each view uses R = R_relative @ R_0 and T = R_relative @ T_0 + T_relative, with R_0 = I and T_0 = [0, 0, 3].

ViewR_relativeT_relativeWhat it does
1rotate -90° about z[0, 0, 0] Rolls the camera about its own viewing axis. T_0 lies on that axis and is unchanged, so no translation is needed.
2rotate +90° about y[-3, 0, 3] Swings the camera to the side of the cow. The rotation also carries T_0 to [3, 0, 0], putting the cow out of frame, so T_relative moves it back to [0, 0, 3] in front of the camera.
3identity[0, 0, 2] Moves the camera 2 units further back, so the cow appears smaller.
4identity[0.5, -0.5, 0] Translates the camera within the image plane, shifting the cow toward the lower right at unchanged size.
Camera transform 1
View 1
Camera transform 2
View 2
Camera transform 3
View 3
Camera transform 4
View 4

5. Rendering Generic 3D Representations

5.1 Point Clouds from RGB-D Images

Both views are unprojected into world coordinates, so the union is a plain concatenation of the two point and colour tensors.

Three point clouds of a plant rendered side by side
Left: first view (125,035 points). Middle: second view (123,749 points). Right: the union (248,784 points), which fills in most of the holes left by self-occlusion in either single view.

5.2 Parametric Functions

x = (R + r·cosθ)·cosφ    y = r·sinθ    z = (R + r·cosθ)·sinφ

R = 1.0, r = 0.4, with θ and φ each sampled 200 times over [0, 2π]. y and z are swapped from the textbook form so the hole lies along the y-up axis of PyTorch3D.

Torus point cloud
Torus point cloud, 40,000 points.

New object: a helix tube, obtained by letting the height rise linearly with φ and letting φ wrap three times.

Helix tube point cloud
Helix tube: R = 1.0, r = 0.25, 3 turns.

5.3 Implicit Surfaces

F(x, y, z) = (sqrt(x² + z²) - R)² + y² - r²

F is evaluated on a 128³ voxel grid spanning -1.5 to 1.5, and marching cubes extracts the zero level set, giving 41,024 vertices and 82,048 triangles.

Torus mesh extracted from an implicit function
Torus mesh from marching cubes.

Tradeoffs: mesh versus point cloud

Measured at 256 × 256 on CPU, mean of five renders after a warm-up (three for the plant). Memory counts coordinates and colour as float32 and face indices as int64.

ObjectRepresentationPrimitivesTime / frameMemory
Torus, marching cubesmesh82,048 triangles4.14 s2,884 KB
Torus, parametricpoint cloud40,000 points3.29 s938 KB
Torus, parametric, triangulatedmesh4,096 triangles0.20 s144 KB
Cowmesh5,856 triangles0.29 s206 KB
Plant, both views mergedpoint cloud248,784 points21.65 s6,803 KB

Speed. Meshes are not simply faster. The marching cubes torus is slower than the point cloud of the same shape, because cost tracks primitive count rather than representation. The advantage of a mesh is that it can cover a surface with far fewer primitives: triangulating the same torus from its parametric function needs 4,096 triangles and renders 20 times faster at equal quality. Marching cubes forfeits that advantage by emitting uniform triangle density regardless of curvature.

Quality. Point clouds leave gaps between samples and carry no normals, so there is no real shading and colour must be baked in per point. A mesh has connectivity, so normals follow from the faces and Phong shading works, which is why the 5.3 torus reads as solid while the 5.2 torus looks flat.

Memory. Per element a mesh costs more, entirely because of the face list. At equal visual quality that reverses: 144 KB for the triangulated torus against 938 KB for the point cloud. Extracting a mesh from an implicit function adds a further cost, since 2,097,152 voxel evaluations yield only 41,024 vertices, discarding about 98 percent of the work.

Ease of use. Point clouds are what depth sensors output directly, and 5.1 built one with no topology decisions at all. Meshes need a valid triangulation, and recovering one from points normally requires fitting an implicit function first, but meshes are what texturing, editing and animation tools expect.

A second implicit surface

Two sphere distance functions combined by boolean union, F = min(f1, f2). The spheres have radius 0.7 with centres 1.0 apart, so marching cubes produces one connected surface with a neck. A parametric surface cannot merge shapes this way, because its topology is fixed by the parameter domain.

Union of two spheres
Union of two spheres.

6. Extra Credit: Do Something Fun

Procedural stripes whose phase advances every frame, so the pattern flows along the cow as the camera orbits.

mask = sin(z · 28 + phase) > 0    color = mask · red + (1 - mask) · white
Cow with animated flowing stripes
Phase advances one full period over the 36 frames, so the loop is seamless.

7. Extra Credit: Sampling Points on Meshes

A face is drawn with probability proportional to its area, then a barycentric coordinate is sampled with the square root correction that keeps points uniform rather than clustered toward the centre:

α1 = 1 - sqrt(u)   α2 = sqrt(u)·(1 - v)   α3 = sqrt(u)·v

The point is then α1·p1 + α2·p2 + α3·p3.

Cow sampled at increasing point counts next to the original mesh
From left: 10, 100, 1000 and 10000 sampled points, with the original mesh on the right.