CMU 16-825 • Learning for 3D Vision

Assignment 1
Rendering Basics with PyTorch3D

Camera control, triangle meshes, texture interpolation, RGB-D point clouds, parametric surfaces, implicit surfaces, and creative 3D rendering.

PyTorch3D Mesh Rendering Point Clouds Marching Cubes

Overview

Rendering and 3D representations

This assignment explores the basic PyTorch3D rendering pipeline and several common 3D representations. I use cameras, lights, triangle meshes, per-vertex textures, point clouds, parametric surfaces, and implicit functions to generate the visual results shown below.

1. Practicing with Cameras

360° render and Dolly Zoom

360 degree render of cow mesh

1.1 360-degree Render

I rotate the camera around the cow by varying the azimuth angle while keeping the distance and elevation fixed. The frames are combined into a looping GIF to create a turntable visualization.

Dolly zoom animation

1.2 Dolly Zoom

The field of view and camera distance are varied together so the subject remains approximately the same size while the perspective distortion changes.

d · tan(FOV / 2) ≈ constant

2. Practicing with Meshes

Constructing geometry by hand

Rotating tetrahedron mesh

2.1 Tetrahedron

The tetrahedron is constructed manually from 4 vertices and 4 triangular faces.

Rotating cube mesh

2.2 Cube

The cube uses 8 vertices and 12 triangular faces. Each square side is represented by two triangles.

3. Re-texturing a Mesh

Vertex colors from geometry

Cow mesh with red-blue gradient texture

I use the z-coordinate of each cow vertex to interpolate between two RGB colors. The front of the cow is blue and the back is red.

color1 = [0, 0, 1]   # blue
color2 = [1, 0, 0]   # red
α = (z − zmin) / (zmax − zmin)
color = α · color2 + (1 − α) · color1

4. Camera Transformations

Relative rotation and translation

Each target view is obtained by applying a relative transform to the original camera extrinsics:

R = Rrelative R0      T = Rrelative T0 + Trelative
Original camera pose

Original

Camera transformed by negative 90 degree Z rotation

Z −90°

Camera translated along z

Z Translation

Camera translated along x

X Translation

Camera transformed by positive 90 degree Y rotation

Y +90°

5.1 RGB-D Point Clouds

Unprojecting pixels into 3D

The RGB-D images are unprojected into world-space point clouds using the provided camera intrinsics, extrinsics, depth map, and RGB image. I render the first view, the second view, and their union.

RGB-D point cloud from first image

Point Cloud 1

RGB-D point cloud from second image

Point Cloud 2

Union of RGB-D point clouds

Union

5.2 Parametric Functions

Sampling a 2D domain into 3D

Parametric torus point cloud

Required: Torus Point Cloud

A torus is sampled from its parametric function on a regular 2D grid and rendered as a point cloud. The 360° view makes the central hole visible.

Parametric Mobius strip point cloud

Additional Object: Möbius Strip

As an additional parametric object, I sample a Möbius strip and render the resulting point cloud from continuously changing viewpoints.

5.3 Implicit Surfaces

Voxel fields and Marching Cubes

For implicit surfaces, I evaluate a scalar function on a 3D voxel grid and use Marching Cubes to extract the zero level set as a triangle mesh.

Implicit function → Voxel grid → Marching Cubes → Triangle mesh → Render
Implicit torus mesh

Required: Implicit Torus

The torus is represented by an implicit function and converted to a mesh with Marching Cubes.

Implicit gyroid mesh

Additional Object: Gyroid

The Gyroid demonstrates how an implicit representation can generate complex surfaces without manually defining vertices and faces.

Mesh vs. Point Cloud

Mesh

Explicit connectivity produces continuous surfaces and generally cleaner shading. Meshes are especially convenient for surface rendering, but the topology and face information add complexity and memory overhead.

Point Cloud

Point clouds are simple to construct and manipulate, but rendering quality depends strongly on sampling density and point radius. Sparse point clouds can contain visible holes and do not explicitly encode surface connectivity.

Extra Credit 6

Do Something Fun: Animated Metaballs

Animated metaball implicit surface

I create multiple moving scalar-field centers and extract a new implicit surface at every frame. As the centers move, the components smoothly merge and separate, producing an animated metaball-like structure.

Extra Credit 7

Sampling Points on Meshes

Instead of using the mesh vertices directly, I sample the cow surface using area-weighted face sampling and uniform barycentric coordinates. This avoids over-representing regions composed of many small triangles.

  1. Compute each triangle's area.
  2. Sample a face with probability proportional to its area.
  3. Sample uniform barycentric coordinates inside the chosen triangle.
  4. Convert the barycentric coordinates into a 3D point.
Cow mesh compared with 10 sampled points

10 points

Cow mesh compared with 100 sampled points

100 points

Cow mesh compared with 1000 sampled points

1,000 points

Cow mesh compared with 10000 sampled points

10,000 points

Implementation

Tools used

Python PyTorch PyTorch3D PyMCubes NumPy ImageIO