Assignment 1: Rendering Basics with PyTorch3D

16-825 Learning for 3D Vision · Andrew ID: lnayak

1. Practicing with Cameras

1.1 360 degree renders

360 degree turntable of the cow mesh

1.2 Dolly zoom

Dolly zoom effect on the cow

2. Practicing with Meshes

2.1 Tetrahedron

The mesh has 4 vertices and 4 triangle faces.

360 degree turntable of a tetrahedron

2.2 Cube

The mesh has 8 vertices and 12 triangle faces.

360 degree turntable of a cube

3. Re-texturing a mesh

color1 = [0, 0, 1], blue, at the front of the cow, and color2 = [1, 0, 0], red, at the back.

Cow with a blue to red gradient texture

4. Camera Transformations

ViewR_relativeT_relativeWhat it is doing
190 degrees about z[0, 0, 0] Rolls the camera about its own optical axis, so the cow does not move and the picture rotates. No translation is needed because T_0 lies on the z axis, which the rotation leaves unchanged.
290 degrees about y[-3, 0, 3] Orbits the camera a quarter turn to look at the cow from the side. The rotation alone would move T_0 to [3, 0, 0] and swing the cow out of frame, so the translation puts it back 3 units in front of the camera.
3Identity[0, 0, 2] Moves the camera back from 3 units to 5 units without rotating it, so the cow appears smaller from the same angle.
4Identity[0.5, -0.5, 0] Slides the camera sideways and up without rotating it. In view space +x is left and +y is up, so this shifts the cow right and down in the image.
Original camera view before any transformation

Original (reference): R_relative = identity, T_relative = [0, 0, 0]

Transform 1

View 1

Transform 2

View 2

Transform 3

View 3

Transform 4

View 4

5. Rendering Generic 3D Representations

5.1 Point clouds from RGB-D images

Three plant point clouds side by side

Left: first image. Middle: second image. Right: union of the two.

5.2 Parametric functions

Torus point cloud

Torus point cloud.

Mobius strip point cloud

New object: Mobius strip.

5.3 Implicit surfaces

Torus mesh from an implicit function

Torus mesh.

Tradeoffs between rendering as a mesh and as a point cloud

MeshPoint cloud
Rendering quality Gives a continuous watertight surface, so shading, occlusion and silhouettes are all correct. Gaps show between points, and the point radius has to be tuned to the density to avoid a sparse or a blobby look.
Rendering speed Rasterizing the triangles is fast, but building the mesh first means evaluating the implicit function over the whole voxel grid and running marching cubes. Renders immediately, since there is no connectivity to compute and each point is splatted independently.
Memory usage Stores vertices, faces and normals, and the voxel grid needed to build it grows as the cube of the resolution, so doubling detail costs eight times the memory. Stores only coordinates and colors, which is much lighter for the same visual coverage.
Ease of use Connectivity has to stay valid, and changing the topology of the surface is awkward. Easy to build, merge and subsample. Combining two point clouds is just a concatenation of their points.

Overall, a mesh is the better choice when an actual surface is needed, for lighting, occlusion and watertight geometry. A point cloud is the better choice for flexibility, since it comes directly out of a depth sensor, merges across views for free, and never requires a decision about topology.

Gyroid mesh from an implicit function

New object: gyroid.

6. Extra credit: something fun

Sphere morphing into a torus

A sphere morphing into a torus and back, by blending the two implicit functions.

7. Extra credit: sampling points on meshes

Sampled point clouds next to the cow mesh

Left to right: 10, 100, 1000 and 10000 sampled points, then the original cow mesh.