Assignment 1: Rendering Basics with PyTorch3D

16-825 Learning for 3D Vision · Fall 2026 · Jinxi He (ginh)

1.1 360-degree Renders (5 points)

A turntable render of the provided cow mesh: the camera orbits the origin at distance 3 with look_at_view_transform, sweeping the azimuth over a full revolution (60 frames). A point light is attached to the camera center so every viewpoint is lit.

360-degree render of the cow mesh
360-degree turntable of the cow mesh.

1.2 Re-creating the Dolly Zoom (10 points)

While the field of view widens, the camera dollies in, keeping \( d \cdot \tan(\mathrm{fov}/2) \) constant (anchored at distance 3, fov \(60^\circ\)) so the cow stays the same size in the frame. The subject holds still while the perspective of the ground plane stretches — the Vertigo effect.

Dolly zoom effect
Dolly zoom: the fov sweeps from \(5^\circ\) to \(110^\circ\) while the camera dollies in.

2.1 Constructing a Tetrahedron (5 points)

The tetrahedron is built by hand: an apex above a triangular base, with every face wound counter-clockwise as seen from outside so all normals point outward. It has 4 vertices and 4 triangular faces — any 3 of the 4 vertices span a face, giving \( \binom{4}{3} = 4 \) faces.

360-degree render of a tetrahedron
360-degree turntable of the hand-built tetrahedron (elevation 25 degrees).

2.2 Constructing a Cube (5 points)

The cube uses 8 vertices and 12 triangular faces: each of the 6 square faces is split into 2 triangles along a diagonal. Note the smooth-looking shading — PyTorch3D averages face normals at shared vertices, so the 90-degree edges are rendered as if they were rounded. Making the edges crisp would require duplicating vertices per face (24 vertices) so normals are not averaged across faces.

360-degree render of a cube
360-degree turntable of the hand-built cube.

3 Re-texturing a Mesh (10 points)

Each vertex is colored by linearly interpolating between two colors according to its \(z\) coordinate (front of the cow at \(z_{\min}\), back at \(z_{\max}\)). I chose \( \mathrm{color}_1 = (0.1,\, 0.7,\, 0.9) \) (teal) for the front of the cow (smallest \(z\)) and \( \mathrm{color}_2 = (1.0,\, 0.5,\, 0.1) \) (orange) for the back (largest \(z\)) — a complementary pair that stays bright under Phong shading, so the gradient reads clearly from every viewpoint.

Cow with front-to-back color gradient
Teal head to orange tail, interpolated per vertex along \(z\).

4 Camera Transformations (10 points)

Starting from the base extrinsics \( R_0 = I \), \( T_0 = (0, 0, 3) \) (camera 3 units in front of the cow), each image below is produced by a relative transform with \( R = R_{\mathrm{rel}} R_0 \) and \( T = R_{\mathrm{rel}} T_0 + T_{\mathrm{rel}} \). In PyTorch3D's convention, \( R_{\mathrm{rel}} \) rotates the world as seen from the old camera, and \( T_{\mathrm{rel}} \) translates it in camera coordinates (\(+x\) is screen-left, \(+y\) is up, \(+z\) is away from the camera).

Base view
Base view: \( R_{\mathrm{rel}} = I \), \( T_{\mathrm{rel}} = (0,0,0) \).
Camera rolled 90 degrees
Transform 1 — roll. A pure rotation of the camera about its own optical axis by 90 degrees (no translation): the image spins in place while framing is unchanged.
Camera moved back
Transform 2 — dolly back. A pure translation \( T_{\mathrm{rel}} = (0,0,2) \) pushes the world further along the viewing axis (camera distance 3 to 5), shrinking the cow.
Camera shifted in the image plane
Transform 3 — in-plane shift. A pure translation \( T_{\mathrm{rel}} = (0.5, -0.5, 0) \) in camera coordinates slides the cow screen-left and screen-down without changing its size.
Camera orbited to the side
Transform 4 — 90-degree orbit. \( R_{\mathrm{rel}} \) rotates the camera 90 degrees about the world \(y\) axis to the cow's side; since rotation alone would also rotate \(T_0\), \( T_{\mathrm{rel}} = (-3,0,3) \) cancels that and keeps the camera looking at the origin from distance 3. (Dimmer because the point light stays fixed.)

5.1 Rendering Point Clouds from RGB-D Images (10 points)

Each RGB-D image is unprojected with unproject_depth_image: every masked pixel is cast along its viewing ray to its depth value using the camera intrinsics and extrinsics, producing a colored 3D point in world space. Because both cameras place their points in the same world frame, the union of the two clouds is a plain concatenation — no alignment step is needed.

Point clouds from two RGB-D images and their union
Left: cloud from image 1. Middle: cloud from image 2. Right: their union. Each single view has holes where the plant occludes itself; the union fills in the missing regions and is visibly denser.

5.2 Parametric Functions (10 + 5 points)

A torus point cloud is sampled from its parametric equation (ring radius \(R = 1\), tube radius \(r = 0.4\)): \[ x = (R + r\cos\theta)\cos\phi, \qquad y = r\sin\theta, \qquad z = (R + r\cos\theta)\sin\phi, \] with a \(200 \times 200\) grid over \( (\theta, \phi) \in [0, 2\pi)^2 \). The torus lies flat so the hole points up and stays visible from the elevated (30-degree) orbit. Points are colored by their normalized \(xyz\) position. As with the sphere example below, the sample count controls apparent surface density.

Sphere sampled 100x100
Sphere, \(100 \times 100\) samples.
Sphere sampled 1000x1000
Sphere, \(1000 \times 1000\) samples.
Torus point cloud
Torus point cloud (\(200 \times 200\) samples), hole visible throughout the orbit.
Klein bottle point cloud
New object: a Klein bottle (classic bottle-shaped immersion, \(400 \times 400\) samples) — a closed surface with no inside or outside; the neck loops over and passes through the side of the bottle.

5.3 Implicit Surfaces (15 + 5 points)

The torus is defined implicitly by \[ F(x,y,z) = \left( \sqrt{x^2 + z^2} - R \right)^2 + y^2 - r^2 \] with \(R = 1\), \(r = 0.4\) (hole along \(+y\)). \(F\) is evaluated on a \(96^3\) voxel grid over \([-1.5, 1.5]^3\) and the mesh is extracted as the zero level set with marching cubes, then the vertices are mapped from voxel-index space back to world coordinates. The extracted mesh has 23,280 vertices and 46,560 faces.

Implicit torus mesh
Torus mesh extracted from the implicit function via marching cubes.
Gyroid mesh
New object: a gyroid, the triply periodic minimal surface \( \sin x \cos y + \sin y \cos z + \sin z \cos x = 0 \), extracted over three periods (133k vertices).

Mesh vs. point cloud rendering trade-offs

6 Do Something Fun (+10 points)

A cartoon squash-and-stretch animation built purely by editing vertex positions: the cow's legs (vertices below hip height) stretch downward, then its body vertically squashes while widening sideways. The two deformations are continuous at the hip line, and the sideways bulge ramps in smoothly so the surface never tears. Faces and the UV texture are reused unchanged every frame (Meshes.update_padded), and the sequence ping-pongs into a seamless loop.

Squash and stretch cow animation
Squash-and-stretch: legs stretch, body squashes, then the motion reverses.

7 Sampling Points on Meshes (+10 points)

Uniform surface sampling by stratified sampling: (1) pick a triangle with probability proportional to its area; (2) draw a uniform barycentric coordinate inside it using the square-root trick, \( (w_0, w_1, w_2) = (1 - \sqrt{u},\ \sqrt{u}(1 - v),\ \sqrt{u}v) \), which avoids the clustering that naive uniform weights produce; (3) the sample is the barycentric combination of the face's corners. Sampling faces by area keeps the density even where triangle sizes vary.

Cow sampled with increasing point counts next to the mesh
Left to right: 10, 100, 1000, 10000 sampled points, and the original mesh. 10 points are unrecognizable, 1000 already read as a cow, and 10000 approach a solid surface — while the mesh stays smooth at any scale.