16-825 Assignment 2: R

This page includes text, an image, and GIFs generated using trained models.

Task 1.1 — voxel

Result:

GIF 1
src
GIF 2
tgt

Task 1.2 — point cloud

Result:

dolly zoom
src
dolly zoom
tgt

Task 1.3 — mesh

Result:

dolly zoom
src
dolly zoom
tgt

Task 2.1 — voxel pred

3 examples:

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted voxel

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted voxel

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted voxel

Task 2.2 — point cloud pred

3 examples:


tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted point cloud

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted point cloud

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted point cloud

Task 2.3 — mesh pred

3 examples:


tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted mesh

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted mesh

tetrahedron
ground truth
tetrahedron
input rgb
tetrahedron
predicted mesh

Task 2.4 — Quantitative comparisions

Result:

voxel
points
mesh

Task 2.5 — Hyperparameter

Changed start mesh to a cow. My predictions are all cow flavored (aka predictions all have cow-like features/topologies).

Task 2.6 — Interpret your model

Result:

One paper I looked at is https://github.com/nywang16/Pixel2Mesh and tried to implement it for point clouds.

point alt visualization
ground truth mesh
pred point cloud
input rgb

Task 3 — Exploring other architectures / datasets.

I chose to Extended dataset for training, by training points with all 3 classes

This result makes sense because the topology of a plane, car, and chair are very different. The resulting blob is due to the model needing to learn one latent mapping to 3 different topologies. The blob indicates that our model learned an average of the three topologies. I'm not sure if theres a classification associated with each of the models in the dataset or if we learn based on a classification (although i see no indication of it in the code), but including class label may help with this blob situation.

cow
ground truth
cow
gt rgb image
cow
pred pointcloud