Shown below are the ground truth voxel grid (left), as well as the fitted voxel grid (right) after optimizing over the loss function.

Shown below are the ground truth point cloud (left), as well as the fitted point cloud (right) after optimizing over the loss function.

Shown below are the ground truth mesh (left), as well as the fitted mesh (right) after optimizing over the loss function.

Since voxels represent 3D imagery with a pre-defined structure, a Convolutional Neural Network (CNN) is the perfect candidate. In model.py, I created a 3D CNN with 4 convolutional layers, and with ReLUs in between. I also added in a sigmoid at the evaluation point, since that’s what the logits-based loss function uses by default. Below are a few examples. For each example, on the left is the original image, in the middle is the ground truth, and on the right is the predicted 3D structure.



The average F1@0.05 score was 35.937. The F1 plot is shown below.
Unlike voxels, point clouds don’t have a predefined structure and can be in any order. Thus, I used a simple deep network with 3 linear layers, of 512, 1024, and 2048 nodes in each hidden layer. Each linear layer is also separated by a LeakyReLU, with a Tanh function used at the output layer for normalization. Below are a few examples. For each example, on the left is the original image, in the middle is the ground truth, and on the right is the predicted 3D structure.



The average F1@0.05 score was 80.084. The F1 plot is shown below.
Similar to point clouds, meshes don’t have a predefined structure and can be in any order. Thus, I used the exact same DNN structure that I described for point clouds. Below are a few examples. For each example, on the left is the original image, in the middle is the ground truth, and on the right is the predicted 3D structure.



The average F1@0.05 score was 70.897. The F1 plot is shown below.