Assignment 1: Rendering Basics with PyTorch3D

16-825 Learning for 3D Vision

1. Practicing with Cameras

1.1 360-degree Cow Render

A cow mesh rendered while the camera moves through 360 degrees
The azimuth changes from -180 degrees to 180 degrees while elevation and distance remain fixed.

1.2 Dolly Zoom

Dolly zoom animation of a cow on a plane
The field of view and camera distance change together to keep the cow at approximately the same image size.

2. Practicing with Meshes

2.1 Tetrahedron

A tetrahedron rendered from all sides
360-degree render of a manually defined tetrahedron.

Vertices: 4
Triangle faces: 4

2.2 Cube

A cube rendered from all sides
360-degree render of a manually defined cube.

Vertices: 8
Triangle faces: 12

3. Retexturing a Mesh

I chose blue as color1 = [0, 0, 1] and red as color2 = [1, 0, 0].

For every vertex, its z-coordinate is normalized to obtain an interpolation value called alpha. The vertex color is then computed as:

color = (1 - alpha) * color1 + alpha * color2

Vertices with the smallest z-coordinate are blue, vertices with the largest z-coordinate are red, and the values between them form a smooth gradient.

Cow mesh with a blue-to-red color gradient
Retextured cow rendered through 360 degrees.

4. Camera Transformations

R_relative changes the camera orientation, while T_relative changes the translation used to express world points in camera coordinates.

Transform 1

R_relative = [[ 0, 1, 0],
              [-1, 0, 0],
              [ 0, 0, 1]]
T_relative = [0, 0, 0]

This rotates the camera coordinate frame by -90 degrees around the z-axis, so the cow appears sideways.

Cow rendered with camera transform 1

Transform 2

R_relative = [[ 0, 0, 1],
              [ 0, 1, 0],
              [-1, 0, 0]]
T_relative = [-3, 0, 3]

This rotates the camera around the y-axis to produce a side view. The translation keeps the cow visible and approximately centered.

Cow rendered with camera transform 2

Transform 3

R_relative = identity
T_relative = [0, 0, 2]

The additional z translation increases the effective distance between the camera and cow, so the cow appears smaller.

Cow rendered with camera transform 3

Transform 4

R_relative = identity
T_relative = [0.5, -0.5, 0]

The x and y translations move the cow horizontally and vertically in the rendered image.

Cow rendered with camera transform 4

5. Rendering Generic 3D Representations

5.1 RGB-D Images to Point Clouds

Each valid depth pixel is unprojected through its camera to create a 3D point. Its RGB value becomes the feature of that point. The union concatenates the points and colors from both views.

Point cloud reconstructed from the first RGB-D view
Point cloud from view 1
Point cloud reconstructed from the second RGB-D view
Point cloud from view 2
Union of point clouds reconstructed from two RGB-D views
Union of both point clouds

5.2 Parametric Functions

Torus point cloud:

major_radius = 1.0
minor_radius = 0.4

x = (major_radius + minor_radius * torch.cos(Theta)) * torch.cos(Phi)
y = minor_radius * torch.sin(Theta)
z = (major_radius + minor_radius * torch.cos(Theta)) * torch.sin(Phi)

My custom object, a cone:

radius = (1 - Height) / 2

x = radius * torch.cos(Theta)
y = Height
z = radius * torch.sin(Theta)
Torus represented as a parametric point cloud
Parametric torus point cloud
Cone represented as a parametric point cloud
Parametric cone point cloud

5.3 Implicit Surfaces

I evaluated each function on a voxel grid and used Marching Cubes to create a mesh. Torus function:

major_radius = 0.7
minor_radius = 0.3

voxels = (torch.sqrt(X**2 + Z**2) - major_radius)**2 \
         + Y**2 - minor_radius**2

My custom object, an ellipsoid:

radius_x = 0.6
radius_y = 1.0
radius_z = 0.6

voxels = (X / radius_x)**2 \
         + (Y / radius_y)**2 \
         + (Z / radius_z)**2 - 1
Torus mesh extracted from an implicit function
Implicit torus mesh
Ellipsoid mesh extracted from an implicit function
Implicit ellipsoid mesh

Point Cloud and Mesh Trade-offs

Representation Advantages Disadvantages
Point cloud Simple to create and directly represents sampled surface locations. Has no surface connectivity; sparse sampling may create visible gaps.
Mesh Provides connected surfaces and supports smooth lighting and shading. Requires faces and valid topology; extracting a mesh adds computation.

Extra Credit

6. Creative Scene: Disco Cow

I created animated cyan and magenta stripes on the cow. The stripe color changes with vertex height and animation phase while the camera completes a 360-degree orbit.

Cow with animated cyan and magenta disco stripes
Animated disco cow

7. Sampling Points from a Mesh

Triangles are selected with probability proportional to their surface area. Points are then sampled uniformly inside each selected triangle using barycentric coordinates. The comparison below shows the original cow mesh and point clouds containing 10, 100, 1,000, and 10,000 points.

Cow mesh compared with samples of 10, 100, 1000, and 10000 points
Original mesh and increasingly dense point-cloud samples