Assignment 1 — Learning for 3D Vision

16-825, Fall 2026 · Zijin Ye (Andrew ID: zijiny)

Rendering basics with PyTorch3D: turntables, camera extrinsics, hand-built polyhedra, and three ways of writing down the same torus. Every figure below is regenerated by run_all.py into output/.

1.1 360-degree Renders (5 points)

36 cameras at distance 3 and elevation 0, with azimuth swept evenly over [0°, 360°) using look_at_view_transform. The 360° frame is dropped so the loop does not stutter.

The point light rides along with the camera instead of sitting at a fixed world position. The starter code already does this for its single view — camera centre and light are both at (0, 0, −3) — and keeping them together is what stops the far side of the cow from going black halfway through the turn.

The cow mesh rotating through a full 360-degree turn
36 frames at 256 px, 15 fps. starter/render_360.py

1.2 Re-creating the Dolly Zoom (10 points)

A subject of half-height h at the origin covers a constant fraction of the frame as long as tan(fov / 2) = h / distance. Sweeping the field of view from 5° to 120° and solving for the distance at each step cancels the change in focal length:

h        = 3 · tan(30°) = 1.732      # so distance == 3 when fov == 60°
distance = h / tan(fov / 2)
T        = [[0, 0, distance]]

Across the 40 frames the camera travels from 39.7 units away at 5° to 1.0 units away at 120°. The cow holds its size while the checkerboard behind it stretches out and swallows the frame.

Dolly zoom: the cow stays the same size while the checkered plane stretches
The current field of view is printed on each frame. starter/dolly_zoom.py

2.1 Constructing a Tetrahedron (5 points)

Mesh size

A tetrahedron needs 4 vertices and 4 triangular faces.

The vertices are four alternating corners of the cube [−1, 1]³, which makes the tetrahedron regular. Each face is wound counter-clockwise seen from outside, so every face normal points away from the centre:

vertices = (1, 1, 1), (1, −1, −1), (−1, 1, −1), (−1, −1, 1)
faces    = (0, 1, 2), (0, 3, 1), (0, 2, 3), (1, 3, 2)

These renders use a flat shader rather than the Phong shader in utils.py: with only four vertices, per-vertex normals average across all three faces meeting at a corner and smear the solid into a soft blob. One normal per face keeps the facets crisp.

A tetrahedron rotating through 360 degrees
Vertices tinted along y so the edges stay legible. starter/simple_meshes.py

2.2 Constructing a Cube (5 points)

Mesh size

A cube needs 8 vertices and 12 triangular faces — two triangles per square side.

The eight corners of [−1, 1]³, with each square side split into two triangles sharing a diagonal. As with the tetrahedron, all twelve triangles are wound counter-clockwise from outside, so the six outward normals come out as ±x, ±y, ±z.

A cube rotating through 360 degrees
Same flat shader and y-tint as the tetrahedron. starter/simple_meshes.py

3. Re-texturing a Mesh (10 points)

Colour choice

color1 = [0.0, 0.75, 1.0] — cyan, at the front of the cow (smallest z).
color2 = [1.0, 0.15, 0.55] — magenta, at the back (largest z).

Every other vertex is a linear blend of the two, weighted by where its z falls between z_min and z_max:

alpha = (z − z_min) / (z_max − z_min)
color = alpha · color2 + (1 − alpha) · color1

The pair was picked for maximum separation in hue while keeping both ends bright enough to survive the Phong shading, so the nose-to-tail ramp stays readable from every angle in the turn.

The cow with a cyan-to-magenta gradient from front to back, rotating
Per-vertex colours interpolated along the z axis. starter/retexture.py

4. Camera Transformations (10 points)

PyTorch3D maps world points into camera space with X_cam = X_world @ R + T. Two consequences make each target image readable by inspection. First, row i of R is where the world basis vector ei ends up in camera coordinates. Second, an object at the world origin lands at exactly X_cam = T, so T alone says where the subject sits in frame and how far away it is.

Camera space has +X pointing left in the image, +Y up and +Z into the screen. Reading which way the coloured axis arrows point in each target therefore pins down the rows of R directly, and the requirement that a centred subject sits at T = (0, 0, 3) pins down T_relative. The arrows are red for world +X, green for +Y and blue for +Z.

transform1 — roll the camera

Target: cow rotated 90 degrees clockwise
Target
My render: cow rotated 90 degrees clockwise
My render

The red arrow has swung from left to up and the green from up to right, so the whole image has rotated 90° clockwise. That is the camera rolling −90° about its own optical axis. The camera never moves, so T is unchanged.

R_relative = [[ 0, 1, 0],
              [−1, 0, 0],
              [ 0, 0, 1]]
T_relative = (0, 0, 0)

transform2 — orbit to a side view

Target: cow seen from the side
Target
My render: cow seen from the side
My render

Red has vanished (world +X now points into the screen) and blue points right, so the camera has orbited 90° about the y axis. That rotation alone would send T to (3, 0, 0) and push the cow out of frame, so T_relative pulls it back to (0, 0, 3).

R_relative = [[ 0, 0, 1],
              [ 0, 1, 0],
              [−1, 0, 0]]
T_relative = (−3, 0, 3)

transform3 — dolly straight back

Target: a smaller cow
Target
My render: a smaller cow
My render

Orientation is untouched — the arrows point exactly where they did at the start — and the cow is half the size. Pure translation along the viewing axis, doubling the distance from 3 to 6. Note this is not a zoom: the field of view is unchanged, the camera physically moved.

R_relative = I
T_relative = (0, 0, 3)

transform4 — slide in the image plane

Target: cow shifted left and down
Target
My render: cow shifted left and down
My render

Same orientation, same size, but the cow has slid left and down. Because camera +X is image-left and +Y is image-up, that is a translation of (+0.5, −0.5, 0) — the camera moving right and up by half a unit.

R_relative = I
T_relative = (0.5, −0.5, 0)

These renders use data/cow_with_axis.obj so they line up against the targets arrow for arrow. The same transforms applied to the plain data/cow.obj that camera_transforms.py loads by default are in output/q4_transform_*.jpg.

5.1 Rendering Point Clouds from RGB-D Images (10 points)

Each image is unprojected with unproject_depth_image, which casts a ray through every pixel and stops it at that pixel's depth. Two details needed handling first:

Three plant point clouds side by side: from image 1, from image 2, and their union, all rotating
Left to right: image 1 (62,649 points), image 2 (102,187 points), and the union (164,836 points). Cameras 6 units out, evenly spaced azimuth. Each view covers only the part of the plant its camera could see — the union fills in the leaves hidden from the other. starter/rgbd_pointcloud.py

5.2 Parametric Functions (10 + 5 points)

The torus is sampled on a 400 × 400 grid of (θ, φ) — 160,000 points — with major radius 1.0 and minor radius 0.4:

x = (R + r·cos θ) · cos φ
y =      r·sin θ
z = (R + r·cos θ) · sin φ

Putting the axis of revolution along y rather than z is what keeps the hole visible: a turntable sweeps azimuth, so a torus standing on the y axis presents the same profile from every direction. The camera is tilted 40° above the plane of the ring, enough that the near rim never covers the hole.

A rainbow torus point cloud rotating, hole visible
Torus. Hue runs around the ring; brightness follows the tube angle.
A trefoil knot tube point cloud rotating
New object: a trefoil knot, 54,000 points.

The new object: a trefoil knot

The second object sweeps a circle of radius 0.28 along the trefoil c(t) = (sin t + 2 sin 2t, cos t − 2 cos 2t, −sin 3t), sampled at 600 points along the curve and 90 around each ring. The circle needs a frame to live in, and the textbook Frenet frame flips wherever the curve's curvature vanishes. Instead the normal is parallel transported: start with any vector perpendicular to the first tangent, then at each step re-project the previous normal into the plane perpendicular to the new tangent. The tube comes out with no twist and no pinch anywhere along the knot.

5.3 Implicit Surfaces (15 + 5 points)

The same torus, this time as the zero level set of a single function, recovered with marching cubes over a 128³ voxel grid:

F(x, y, z) = (√(x² + z²) − R)² + y² − r²

The grid spans [−1.55, 1.55] on each axis, leaving clearance around the R + r = 1.4 outer radius so the surface closes instead of being cut by the boundary. Marching cubes returns vertices in voxel index space, so they are mapped back onto the coordinates the function was evaluated at before the mesh is built. The result is 37,856 vertices and 75,712 faces.

A torus mesh rotating, hole visible
Torus mesh, coloured like the 5.2 point cloud so the two are comparable.
Goursat's tangle cube rotating
New object: Goursat's tangle cube, 309,232 faces from a 192³ grid.

The new object: Goursat's tangle cube

One line of polynomial, x⁴ − 5x² + y⁴ − 5y² + z⁴ − 5z² + 11.8 = 0, produces eight lobes joined by twelve necks — a genus-5 surface. That is the argument for implicit representations in one image: the topology is nowhere in the formula, it falls out of the level set, and writing the same object down as an explicit mesh by hand would be miserable.

Mesh vs. point cloud: the trade-offs

Both representations of the identical torus, rendered along the identical camera path:

Side by side: the torus as a point cloud and as a mesh, rotating together
Watch the silhouette and the inner rim: the splats stipple and break up along the edge where the mesh stays continuous.
 Point cloudMesh
Primitives160,000 pts75,712 faces
Construction0.026 s0.105 s
Render, GPU (per frame)17.4 ms12.8 ms
Render, CPU (per frame)29.8 s10.8 s
Geometry in memory3.66 MiB1.73 MiB

Measured over 36 frames at 256 px on an RTX 5070 Laptop, PyTorch3D 0.7.9.

6. Do Something Fun (extra credit, 10 points)

A point cloud that morphs from the cow into a torus and back. The cow mesh is turned into 25,000 surface points with the stratified sampler from task 7; the torus is sampled from the parametric form in 5.2; and the two clouds are put into correspondence by sorting both by azimuth, so the transition unwinds instead of scattering.

Every frame interpolates position and colour between the matched pairs, then applies a twist about the y axis whose angle is proportional to each point's height. The twist is scaled by sin(πα), so it peaks halfway through and vanishes at both ends — the cow and the torus are each seen undistorted. The blend factor follows α = ½ − ½cos(2πt) while the camera makes exactly one revolution, which makes the whole loop seamless.

A point cloud morphing from a cow into a torus and back while rotating
60 frames at 512 px, 25,000 points, 20 fps. starter/fun.py

7. Sampling Points on Meshes (extra credit, 10 points)

Treating a mesh's vertices as a point cloud over-represents wherever the tessellation happens to be dense. Stratified sampling fixes that in three steps:

  1. Pick a face with probability proportional to its area, computed as ½‖(v₁−v₀) × (v₂−v₀)‖ and fed straight to torch.multinomial.
  2. Pick barycentric coordinates uniformly. Drawing (u, v) uniformly from the unit square and folding the half where u + v > 1 back across the diagonal — (1−u, 1−v) — gives a uniform distribution over the triangle, with no rejection and no wasted draws.
  3. Evaluate (1−u−v)·v₀ + u·v₁ + v·v₂.
The cow mesh beside point clouds of 10, 100, 1000 and 10000 sampled points, all rotating
The original mesh (5,856 faces) beside 10, 100, 1,000 and 10,000 sampled points. Colour is position, shared across all five panels. By 1,000 points the silhouette is recognisable; by 10,000 the cloud is hard to tell from the mesh at this resolution. The two smallest clouds are drawn with a larger splat radius so that ten points remain visible at 256 px. starter/sample_points.py