Homework3 learning for 3D

Contact Information:

• Name: Shibo Zhao
• Email: shiboz@andrew.cmu.edu

ipython nbconvert --to HTML your_notebook.ipynb

1.3. Ray sampling (10 points)

You can run the following command to get the result.

python main.py --config-name=box

Visulization

Grid	Rays

1.4. Point sampling (10 points)

You can run the following command to get the result.

python main.py --config-name=box

Visualization

Visulization of sample point

1.5. Volume rendering (30 points)

You can run the following command to get the result.

python main.py --config-name=box

Visualization

Spiral Rendering of Part 1	depth

2.1. Random ray sampling (5 points)

Please check the following function in the code

xy_grid = get_random_pixels_from_image(cfg.training.batch_size, image_size, camera)

2.2. Loss and training (5 points)

Please check the following function in the code

loss = None

Result

Box center: (0.25013208389282227, 0.25044694542884827, -0.0007905613165348768)

Box side lengths: (2.0040059089660645, 1.5028941631317139, 1.5035169124603271)

2.3. Visualization

Please check the following function in the code

python main.py --config-name=train_box

3. Optimizing a Neural Radiance Field (NeRF) (30 points)

Visualization

You can get the result with following command.

python main.py --config-name=nerf_lego

4. NeRF Extras (Choose at least one! More than one is extra credit)

4.1 View Dependence (10 pts)

You can get the result with following command.

python main.py --config-name=nerf_lego

implicit_function:
  type: nerf
  view_dependence: True
  n_harmonic_functions_xyz: 6
  n_harmonic_functions_dir: 2
  n_hidden_neurons_xyz: 128
  n_hidden_neurons_dir: 64
  density_noise_std: 0.0
  n_layers_xyz: 6
  append_xyz: [3]

Turn on View Dependence	Turn off Videw Dependence

NeRF model uses a function of postion and direction to predict the volume density and color of features. If we add more direction to view the scenes (for example: getting more input images to train the network), we will have more chance to learn more detail of scenes. However, if we just get a lot of inputs along a particular direction, the model would overfit to this view. Therefore, the view-dependence should make sure that the modle could learn all direction of the scene and capture all the details.

4.3 High Resolution Imagery (10 pts)

You can get the result with following command.

python main.py --config-name=nerf_lego_highres

I changed the number of points_per_ray with 32, 64 and 128. As expected, the more the number of the sample points per ray, the more detail modle will learn. However, it also increases the memory and comput.

n_pts_per_ray=16	n_pts_per_ray=32	n_pts_per_ray=64	n_pts_per_ray=128