16-825 · Learning for 3D Vision

Assignment 1: Rendering Basics with PyTorch3D

Cameras, meshes, point clouds, and implicit surfaces, rendered with PyTorch3D. Every section below links to the exact script that produced it in starter/.

1.1 · 360-degree Renders

5 pts

Code: starter/q1_1.py · Run: python -m starter.q1_1

Rendered the provided cow mesh from 36 evenly spaced azimuth angles (0°–360°) using look_at_view_transform, keeping distance and elevation fixed.

1.2 · Re-creating the Dolly Zoom

10 pts

Code: starter/dolly_zoom.py · Run: python -m starter.dolly_zoom --num_frames 30

Recreated the dolly zoom effect by moving the camera while changing its field of view, keeping the subject the same apparent size. The camera distance is computed as distance = dolly_constant / tan(fov / 2), so that distance * tan(fov/2) (and hence the cow's apparent size) stays constant while fov sweeps from 5° to 120°. dolly_constant = 1.7 was chosen so the cow fills a reasonable portion of the frame, and znear was lowered to 0.1 so the camera doesn't clip through the mesh once it gets close at wide fov values.

2.1 · Constructing a Tetrahedron

5 pts

Code: starter/q2_1.py · Run: python -m starter.q2_1

Manually defined 4 vertices (alternating corners of a cube) and 4 triangular faces to form a regular tetrahedron, then rendered a 360-degree turntable.

4 vertices4 faces

2.2 · Constructing a Cube

5 pts

Code: starter/q2_2.py · Run: python -m starter.q2_2

Manually defined 8 vertices (cube corners) and 12 triangular faces (each of the 6 square faces split into 2 triangles), then rendered a 360-degree turntable.

8 vertices12 faces

3 · Re-texturing a Mesh

10 pts

Code: starter/q3.py · Run: python -m starter.q3

Re-colored the cow mesh with a gradient based on each vertex's z-coordinate. For every vertex, alpha = (z - z_min) / (z_max - z_min) is computed relative to the mesh's global z range, and the vertex color is set to alpha * color2 + (1 - alpha) * color1. Chose color1 = [0, 0, 1] (blue) for the front of the cow (smallest z) and color2 = [1, 0, 0] (red) for the back (largest z).

4 · Camera Transformations

10 pts

Code: starter/q4.py · Run: python -m starter.q4

Used data/cow_with_axis.obj (which has red/green/blue arrows for the X/Y/Z axes baked into the mesh) to figure out each relative camera transformation, then converted to R_relative/T_relative using R = R_relative @ R_0, T = R_relative @ T_0 + T_relative with R_0 = I, T_0 = [0,0,3]. Distances were calibrated precisely by measuring the green (Y-axis) arrow's pixel length in the target images and matching it exactly, rather than eyeballing it.

5.1 · Point Clouds from RGB-D Images

10 pts

Code: starter/q5_1.py · Run: python -m starter.q5_1

Unprojected the 2 provided RGB-D images of a plant into point clouds using unproject_depth_image, then built a 3rd point cloud as their union. The raw points came out upside-down, so the point coordinates are flipped (negate y and z) before rendering.

5.2 · Parametric Functions

10 + 5 pts

Code: starter/q5_2.py · Run: python -m starter.q5_2

Sampled a torus point cloud from its parametric equation ((R + r cosθ) cosφ, (R + r cosθ) sinφ, r sinθ) with R = 1.5, r = 0.5, over a 200×200 grid of (θ, φ).

New object +5 pts · Trefoil Knot Tube

Code: starter/q5_2_bonus.py · Run: python -m starter.q5_2_bonus

Sampled a tube around a trefoil knot centerline (x=sin t + 2 sin 2t, y=cos t - 2 cos 2t, z=-sin 3t), sweeping a small circle around each point of the curve using a tangent-aligned local frame (tangent from finite differences, normal/binormal from cross products).

5.3 · Implicit Surfaces

15 + 5 pts

Code: starter/q5_3.py · Run: python -m starter.q5_3

Defined a torus implicitly as the zero level-set of F(x,y,z) = (sqrt(x^2+y^2) - R)^2 + z^2 - r^2, evaluated it on a 643 voxel grid, and extracted the surface mesh with marching cubes.

Mesh vs. point cloud trade-offs
  • Surface quality — the mesh is a solid, continuous surface with real normals, so it shades correctly under lighting and looks right at any zoom level or camera distance. Point clouds have no true surface, so they look flatter and show visible gaps when zoomed in or viewed edge-on.
  • Generation cost — building the mesh means evaluating the implicit function on the entire 3D voxel grid and then running marching cubes, so cost scales with voxel resolution and thin or high-curvature features can be missed if the grid isn't fine enough. Sampling a point cloud is much cheaper since you only evaluate the function where you want points.
  • Updating — a point cloud is trivial to add to or resample. A mesh needs the whole grid re-evaluated and re-triangulated to change.
  • Memory — a mesh's connectivity lets it interpolate between vertices, so it needs far fewer elements for the same visual density. A point cloud has no connectivity, so it needs many more points to look equally dense, using more memory per unit of surface.
  • Rendering speed — point rendering is simpler per-primitive, since there's no rasterizing triangles or interpolating shading across a face, so a sparse point cloud can render faster, but a mesh generally reaches equivalent visual quality with far fewer primitives.

New object +5 pts · Metaballs

Code: starter/q5_3_bonus.py · Run: python -m starter.q5_3_bonus

Defined the field as a sum of 3 Gaussian "blobs" centered at different points and thresholded it (field > 0.6), which smoothly merges the individual spheres into a single organic, blobby surface where they overlap — a simple metaball implicit surface.


6 · Do Something Fun

extra credit · +10 pts

Code: starter/q6.py · Run: python -m starter.q6

Built a "carousel" of 8 cow meshes arranged in a ring, each rotated to face outward and recolored with a different hue spaced evenly around the color wheel (via colorsys.hsv_to_rgb). All 8 cows are combined into a single Meshes by concatenating their vertices/faces/textures (offsetting face indices per copy), then rendered orbiting from above.

7 · Sampling Points on Meshes

extra credit · +10 pts

Code: starter/q7.py · Run: python -m starter.q7

Implemented stratified uniform sampling of a triangle mesh's surface:

  • Compute each face's area from its two edge vectors (half the magnitude of their cross product).
  • Sample a face index with probability proportional to its area (torch.multinomial).
  • Sample a uniform random barycentric coordinate on that face: draw u, v ~ Uniform(0,1), and if u + v > 1 reflect with u, v = 1-u, 1-v to fold the point back inside the triangle, keeping the distribution uniform over its area.
  • Compute the sampled point as (1-u-v)*v0 + u*v1 + v*v2.

Rendered the original cow mesh alongside point clouds sampled with 10, 100, 1000, and 10000 points — as the sample count grows the point cloud converges to the mesh's shape.