Setup. The full training runs used a Tesla V100-SXM2-32GB on PSC Bridges-2, PyTorch 2.4.1, PyTorch3D, and NumPy 1.26.4. I first ran small execution checks, then trained NeRF and VolSDF for 250 epochs each and the truck Gaussians for 1,000 iterations. Follow-ups include 500-epoch Lego NeRF at 256 × 256, a four-setting VolSDF sweep, and 3,000-iteration Materials Gaussian comparisons.
A. Neural volume rendering
1. Differentiable Volume Rendering
1.1–1.2 Pipeline
A camera defines each ray’s origin and direction. The sampler places 3D points along the ray, the implicit function returns density and color at these points, and the renderer integrates their contributions into an image. Keeping these operations differentiable allows image-space error to update scene parameters.
1.3 Ray sampling
I generate a pixel grid in [−1, 1], unproject points at depth 1 from NDC into world coordinates, and set every ray origin to the camera center. Each direction is the normalized vector from the camera center to its unprojected point.


1.4 Point sampling
For each ray, I use 64 evenly spaced distances between 0 and 5 and compute
x(t) = o + t d. The resulting positions and distances are stored in the ray bundle.
The visualization uses a regular subset of rays for readability.

1.5 Volume rendering and depth
I compute alpha from density and segment length, use an exclusive cumulative product for transmittance, and sum the weighted colors. The same weights produce depth by replacing color with the distance along the ray.
C = ∑i wi ci · D = ∑i wi ti


2. Optimizing a Basic Implicit Volume
2.1 Random ray sampling
I randomly select 4,096 pixel coordinates without replacement per training image. This reduces the memory needed to optimize the volume compared with rendering all 256 × 256 pixels.
2.2 Loss and training
I optimize the box center and side lengths using mean squared RGB error and Adam at a learning rate of 0.0005. Training uses four posed images over 1,000 outer epochs, with one update per image. The final displayed loss was approximately 0.000001.
| Parameter | Initial value | Optimized value |
|---|---|---|
| Center | (0.00, 0.00, 0.00) | (0.25, 0.25, 0.00) |
| Side lengths | (1.50, 1.50, 1.50) | (2.01, 1.50, 1.50) |
2.3 Before / after and orbit visualization










The fitted volume changes from a centered cube to an offset rectangular box. The recovered parameters are close to a center of (0.25, 0.25, 0) and dimensions (2, 1.5, 1.5). The reference is included for comparison of overall shape, placement, and color pattern.
3. Optimizing a Neural Radiance Field
The base network maps position to density and RGB, using six-frequency positional encoding, a six-layer MLP of width 128 with an input skip at layer index 3, ReLU for density, and sigmoid for color. The base model does not use viewing direction.
For Lego, I use 1,024 rays per batch, 128 samples per ray over distances 2–6, Adam with initial learning rate 0.0005, and 500 epochs at 256 × 256 resolution for the final run. The learning rate is multiplied by 0.8epoch/100. The training objective is RGB MSE. This run starts from scratch, uses rendering chunks of 4,096 rays, and saves orbit renders every 50 epochs and at the final epoch.


The final model recovers the yellow body, dark tracks, raised arm, and supporting platform. At matched display size, the platform texture and outlines of the arm and tracks are more clearly resolved than in the original run, although some thin structures remain imperfect. Both training resolution and training duration changed, so this is not an isolated resolution ablation.
The final validation scores are 24.981 dB PSNR and 0.8681 SSIM. The measured training-plus-render time was 1,032.95 seconds; the complete run took 18 minutes 26 seconds, including setup and evaluation. Raw Lego validation metrics.
4.1 View Dependence
Implemented method. Density depends only on position. A separate RGB head combines the position-network features with a two-frequency encoding of normalized viewing direction. This allows appearance to change between views without making geometry view-dependent.
The comparison uses the Materials scene for both models, 250 epochs, 128 × 128 images, 1,024 rays per batch, 128 samples per ray over distances 1–7, and initial learning rate 0.0001. The view-dependent color head has width 64. Both runs use seed 1.
| Method | PSNR ↑ | SSIM ↑ | Training + render time |
|---|---|---|---|
| Position only | 23.259 dB | 0.8436 | 488.98 s |
| View-dependent color | 24.043 dB | 0.8620 | 569.71 s |
Results and generalization trade-off
Direction conditioning improves validation PSNR by 0.784 dB and SSIM by 0.0185, with approximately 16.5% more measured training-and-render time. The orbit retains the same arrangement of objects in both models; the differences are mainly in their surface brightness and color rather than a large change in silhouette. The view-dependent result has more distinct surface shading on several objects, although the 128 × 128 output limits fine-detail inspection.
Direction conditioning can represent moving highlights and other non-Lambertian appearance, which a position-only model must average across views. The additional flexibility can also fit view-specific training effects that fail to generalize to unseen cameras. Keeping density independent of direction limits one way the model could explain away geometric errors. The color-head architecture also changes, so this is a practical model comparison rather than a parameter-count-matched ablation.
The held-out validation metrics favor the direction-conditioned model in this run. They do not establish that extra view dependence always generalizes better: this comparison uses one seed and one scene. No repeated-run uncertainty estimate was measured.
4.2 Coarse/Fine Sampling
Implemented method. The coarse network evaluates 128 uniformly spaced samples. Its detached rendering weights define a piecewise-constant probability density over depth. Inverse-CDF sampling adds 128 samples, which are merged and sorted with the original positions. A separately parameterized fine network evaluates all 256 positions.
I minimize the sum of coarse and fine RGB MSE. Training uses random PDF samples, while evaluation uses deterministic quantiles. Both networks use the view-dependent architecture from 4.1. The single-network 4.1 result is the baseline, keeping dataset, epochs, and ray batch size fixed.

| Method | Network evaluations/ray | PSNR ↑ | SSIM ↑ | Time |
|---|---|---|---|---|
| Uniform, one network | 128 | 24.043 dB | 0.8620 | 569.71 s |
| Hierarchical, two networks | 128 + 256 | 24.118 dB | 0.8651 | 1,573.02 s |
Speed / quality trade-off
The fine pass concentrates samples near depths that contribute to the coarse image. This can improve the integration of thin or localized surfaces. The chosen configuration costs more: it evaluates 384 points across two networks instead of 128 in one, adds PDF sampling and sorting, and stores activations for both networks during training. Benefits therefore need to be evaluated against actual runtime and memory use. A poor coarse distribution can also undersample a missed surface; retaining the original uniform samples provides some coverage.
Measured outcome. Coarse/fine sampling adds 0.075 dB PSNR and 0.0030 SSIM over the single-network view-dependent model, while taking 2.76× as long in the recorded training-and-render interval. The matched orbit frames look very similar; this is a small refinement rather than a dramatic visual improvement. For this scene and budget, the single-network view-dependent model offers the better time/quality trade-off. The hierarchical model has the highest measured quality of the three NeRF variants, but its small advantage has not been tested across multiple seeds. No peak-memory benchmark was recorded.
The timings above come from the same timer in each NeRF run and include periodic rendering, but exclude final validation. They are distinct from total elapsed times.
4.3 VolSDF
I predict signed distance with a six-layer, width-128 MLP and color with a two-layer, width-128 MLP, both using six-frequency positional encoding. Density is obtained from the Laplace CDF of the negated signed distance. RGB MSE is combined with an Eikonal penalty of weight 0.02.
Training uses 250 epochs, 1,024 rays per batch, 128 samples per ray over distances 2–6, 128 × 128 images, and initial learning rate 0.0005. The learning-rate multiplier is 0.8epoch/50. Four fresh runs use seed 1, identical budgets, rendering chunks of 4,096 rays, and final-epoch outputs. The selected color-reconstruction setting is α = 50, β = 0.02.
L = MSE(C, Cgt) + 0.02 · E[(‖∇d‖2 − 1)²]


| α | β | PSNR ↑ | SSIM ↑ | Training + render time | Evidence |
|---|---|---|---|---|---|
| 10 | 0.02 | 21.118 | 0.6713 | 546.08 s | Metrics |
| 10 | 0.05 | 21.216 | 0.7675 | 563.34 s | Metrics |
| 10 | 0.1 | 20.730 | 0.7342 | 507.27 s | Metrics |
| 50 | 0.02 | 23.142 | 0.8194 | 503.04 s | Metrics |
At fixed α=10, simply reducing β does not improve both metrics: β=0.02 is worse than β=0.05, especially in SSIM. At fixed β=0.02, raising α to 50 improves PSNR by 2.024 dB and SSIM by 0.1482. The higher interior density makes the volume more opaque over a shorter distance, while the narrow transition can localize appearance near the boundary. The selected model exceeds the default α=10, β=0.05 by 1.926 dB and 0.0519 SSIM.
The selected color views have more distinct tracks and platform texture. However, all four extracted meshes contain extensive extraneous surfaces and grid-like artifacts. I select α=50, β=0.02 for its validation color scores, not as evidence of an accurate recovered surface. These are single-seed runs selected on validation, not an independent test-set comparison.
Geometry across all four settings





What do α and β control?
α controls the density scale. Increasing it makes the interior more opaque over a shorter ray segment. β controls the transition width around the surface. With the convention d < 0 inside, density approaches α inside and zero outside; at the surface it is α/2.
How does high versus low β bias the learned SDF?
A large β spreads opacity across a wider band around the zero-level set, allowing a diffuse volume to explain the image and weakening localization of a sharp boundary. A small β concentrates the transition near the surface, favoring sharper geometry, but making the image more sensitive to surface position and to whether the sampling captures that narrow transition.
Which is easier to train?
A larger β is generally easier early in training because more samples lie in the transition region and receive useful gradients. With a very small β, many samples are in nearly saturated regions, and coarse ray sampling may miss the informative band.
Which is more likely to yield an accurate surface?
A smaller β can localize the surface more precisely once the geometry is reasonably initialized and sampling is adequate. It is not automatically better: optimization instability or insufficient sampling can outweigh the benefit. A broad-to-narrow β schedule is a possible follow-up, but was not used in this run.
B. 3D Gaussian splatting
5. 3D Gaussian Rasterization
5.1–5.2 Projection and Gaussian evaluation
For anisotropic Gaussians, I construct the world covariance from rotation and scale as Σ = R S ST RT; the isotropic case reduces to s²I. The camera transform and projection Jacobian map the covariance to the image plane. I project means into screen coordinates and evaluate the quadratic exponent −½(x − μ)TΣ−1(x − μ). The supplied unit tests passed 4/4.
5.3–5.5 Sorting, compositing, and rendering
I filter out Gaussians behind the camera, sort the remaining Gaussians by increasing depth, and compute alpha as opacity times the projected Gaussian value. Exclusive transmittance weights blend color, depth, and silhouette. The final 256 × 256 renders use batches of 256 Gaussians, carrying the remaining transmittance into the next batch.
Native-resolution final renders: the chair was re-rendered at 256 × 256 for both DC-only and spherical-harmonic colors, using the supplied pretrained Gaussians without retraining. Each RGB/depth/mask panel now contains a native 256 × 256 render; these replace the earlier 64 × 64 previews. Both 32-view render passes together took 43 minutes 48 seconds.
Covariance-convention diagnostic
The default implementation uses W=R in J W Σ Wᵀ Jᵀ and passes the four supplied tests.
PyTorch3D transforms row-vector points as xview=xworldR+T;
the corresponding column-vector covariance rotation is therefore Rᵀ.
I added an opt-in --transpose_view_rotation path and checked its covariance
against an independently differentiated projection Jacobian. The default remains unchanged
for test compatibility. This is a separate diagnostic, not an undocumented replacement.
PyTorch3D camera convention.


The tilted view shows substantially cleaner legs and carved edges with the transposed rotation; the stretched streaks in the default rendering largely disappear. Frontal views change much less. No Gaussian parameters were retrained. These observations support the convention diagnosis, but do not replace the assignment's supplied-test results.
Full diagnostic chair orbits: transposed rotation


6. Training 3D Gaussian Representations
I initialize isotropic Gaussians from the provided truck point cloud and optimize their means, scales, colors, and opacities using L1 image loss. Quaternion optimization is unnecessary for isotropic Gaussians. Training uses Adam for 1,000 iterations, a black background, and 2,048 Gaussians per splatting batch.
| Parameter | Learning rate |
|---|---|
| Means | 0.00016 |
| Scales | 0.005 |
| Colors | 0.0025 |
| Opacities | 0.05 |
| Run | Iterations | PSNR ↑ | SSIM ↑ |
|---|---|---|---|
| Execution check | 2 | 17.106 | 0.792 |
| Full training | 1,000 | 29.087 | 0.936 |
The completed full run took 21 minutes 23 seconds, including rendering all 197 training views and evaluation. It improved both held-out metrics over the two-iteration execution check. The last displayed training loss was 0.010.
7.1 Rendering Using Spherical Harmonics
I evaluate the supplied degree-3 spherical harmonics using the normalized direction from the camera center to each Gaussian. All 16 coefficients per color channel contribute to a direction-dependent color, compared with the constant DC color used in section 5. Both runs render the same 32 camera views from the same pretrained chair representation.


Matched RGB comparisons
The slots below show only the left RGB panel of each saved RGB/depth/mask composite. The view index is matched between methods.


View 0. The DC-only backrest is relatively bright and evenly green. SH darkens its side regions and makes the lighter central upholstery pattern stand out more clearly. The silhouette stays the same because only the color evaluation changes.


View 8. In this rotated view, the SH image has a darker seat and stronger contrast between its pale center and green edge. The upright upholstered panel also has stronger directional shading. The geometry and camera are identical, so these changes come from the direction-conditioned appearance. Visible streaks around the legs and thin structures remain in this rotated view; higher resolution does not remove those artifacts.


View 16. Near the returned frontal orientation, the SH backrest again shows darker green regions around a brighter central pattern; the armrests remain bright in both renderings. SH changes appearance rather than repairing geometry or silhouette artifacts. These observations show a shading difference, but without matched ground truth I cannot claim that every SH pixel is more accurate.
7.2 Training on a Harder Scene
Final comparison. All new runs use 10,000 randomly initialized Gaussians, Materials at 128 × 128, seed 1, L1 loss, 2,048 Gaussians per splat, and 3,000 iterations. The baseline retains the truck optimizer and isotropic representation; the original 1,000-iteration Q6-matched baseline is included in the main table and displayed below. The 3,000-iteration baseline controls for the longer training budget of the improved runs.
Selected method: scene-covering initialization, anisotropic Gaussians, revised color/mean learning rates, exponential mean learning-rate decay, and transposed view rotation for covariance projection. No SSIM loss or densification is used.
| Method | Iterations | PSNR ↑ | SSIM ↑ | Total measured time | Evidence |
|---|---|---|---|---|---|
| Required Q6-matched isotropic baseline | 1,000 | 18.002 | 0.6600 | 1,283.40 s | Metrics |
| Isotropic baseline, longer training | 3,000 | 18.667 | 0.6905 | 3,485.00 s | Metrics |
| Improved recipe, W=R | 3,000 | 22.880 | 0.8206 | 3,486.98 s | Metrics |
| Improved recipe, W=Rᵀ (selected) | 3,000 | 23.118 | 0.8357 | 3,478.94 s | Metrics |
Training progress and final renders





Modifications relative to the truck recipe
- Initialization: sample means uniformly inside a radius-1.5 ball, instead of a central Gaussian with standard deviation 0.2. Initialize scale to the root mean squared distance to the three nearest other points and opacity to 0.1, instead of near 1. This spreads the representation across the scene and reduces early occlusion.
- Representation: three scales and a trainable quaternion per Gaussian, rather than one isotropic scale. Projection uses W=Rᵀ in the selected run; the intermediate run keeps W=R to isolate this convention change within the improved recipe.
- Optimization: colors use learning rate 0.01 instead of 0.0025; means start at 0.001 instead of 0.00016 and decay exponentially to 0.00001 over 3,000 iterations. Scale LR remains 0.005, opacity LR 0.05, and quaternion LR is 0.001. The loss remains L1.
- Budget: 3,000 rather than the truck's 1,000 iterations. The equal-length baseline separates the combined recipe's benefit from simply training longer.
Measured outcome. The selected method improves over the equal-length baseline by 4.452 dB PSNR and 0.1451 SSIM. The improved recipe already gains 4.213 dB with W=R; changing only the rotation convention adds a further 0.238 dB and 0.0151 SSIM. Longer baseline training alone raises PSNR from 18.002 to 18.667, much less than the recipe change.
The baseline leaves diffuse outer blobs and broken central objects. The improved models recover separated objects across the scene; the transposed version reduces jagged outlines. Fine reflections and some surfaces remain soft. These are single-seed validation results, and the initialization/learning-rate/representation changes were bundled, so their individual effects cannot be separated. The measured times include visualization and evaluation; small timing differences should not be treated as speed improvements.


Required 1,000-iteration baseline GIFs and initial anisotropy-only experiment
Required baseline. The Materials scene uses 10,000 randomly initialized isotropic Gaussians, the truck learning rates, L1 loss, and 1,000 iterations. The means are sampled from a centered Gaussian distribution with standard deviation 0.2, rather than a reconstructed point cloud. Training images are 128 × 128.
Candidate improvement. The second run uses anisotropic Gaussians with three scale components and trainable quaternion rotations (learning rate 0.001). Dataset, random seed 1, loss, other learning rates, 1,000-iteration budget, and splat batch size 2,048 are held fixed. No densification, SSIM loss, or learning-rate schedule is used in this comparison.
Anisotropic Gaussians can stretch and rotate to fit surfaces with fewer primitives than spheres, but introduce additional parameters and optimization difficulty. Random initialization also makes this problem harder than the truck task: many Gaussians must move before they can explain scene content. The completed comparison shows a small PSNR gain but no consistent improvement across both quality metrics.




| Method | Iterations | PSNR ↑ | SSIM ↑ | Total runtime |
|---|---|---|---|---|
| Isotropic baseline | 1,000 | 18.002 dB | 0.6600 | 1,283.40 s |
| Anisotropic candidate | 1,000 | 18.031 dB | 0.6576 | 1,240.14 s |
Results and limitations
Anisotropy improves PSNR by only 0.029 dB, while SSIM decreases by 0.0024. It is the best of these two runs by PSNR, but the isotropic baseline is better by SSIM; neither dominates. The measured total runtime is about 3.4% lower for the anisotropic run, but this single-run difference is not enough to establish a speed advantage.
The final views expose the main failure: both models have blurry outer blobs and a noisy, incomplete cluster near the center rather than clean, separated objects. The anisotropic run adds elongated streaks in some views without resolving the overall reconstruction. Increasing the representation’s flexibility alone was insufficient under the same 1,000-iteration budget and tightly concentrated random initialization.
Modification summary. Relative to the baseline, the PSNR-best run replaces one isotropic scale with three trainable scales and enables quaternion optimization at learning rate 0.001. All other settings remained fixed in this initial experiment. Its failure motivated the wider initialization, learning-rate changes, and longer controlled runs presented above. Density control was not used in either phase.
Both rows use the Materials validation split. Their timer includes final visualization and evaluation; it differs from the NeRF training-and-render timer. The required baseline metrics are linked in the main comparison table.