Assignment 5

python train.py --task cls --checkpoint_every 10 --main_dir /mnt/data/data/ --checkpoint_dir /mnt/data/checkpoints

test accuracy: 0.9338929695697796

All predition cases for lamps are good.

The failures likely come from global pooling extracting only the most prominent, overlapping features across categories. Shapes with ambiguous traits, like long legs or container-like forms, confusing the model because global pooling provides only coarse, category-level awareness.

python train.py --task seg --checkpoint_every 10 --main_dir /mnt/data/data/ --checkpoint_dir /mnt/data/checkpoints

test accuracy: 0.8924311183144246

All predition cases for lamps are good.

The model performs well when chair parts are clearly separated but struggles on complex shapes where boundaries between components are merged or ambiguous. In these cases, adjacent points with similar local geometry confuse the model, leading to labels spreading into neighboring regions where part boundaries are not clearly defined.

cls:

--num_points 10: test accuracy: 0.3515215110178384

--num_points 50: test accuracy: 0.8908709338929696

--num_points 100: test accuracy: 0.9422875131164743

--num_points 500: test accuracy: 0.9538300104931794

--num_points 1000: test accuracy: 0.9559286463798531

seg:

--num_points 10: test accuracy: 0.6917341977309562

--num_points 50: test accuracy: 0.7752350081037277

--num_points 100: test accuracy: 0.7977795786061588

--num_points 500: test accuracy: 0.8517017828200972

--num_points 1000: test accuracy: 0.8627844408427877

For cls, as long as the number of points is larger then 50, the acc rate will not be affected significitly. For seg, the acc rate is impacted more while decreasing the num_points. But extremely small num_points will not cause a sharp decreasing of acc rate, like when num_points=10.

cls: (num_points=1000)

--rotation_angle 30 --rotation_axis z: test accuracy: 0.5739769150052466

--rotation_angle 45 --rotation_axis z: test accuracy: 0.2864637985309549

--rotation_angle 60 --rotation_axis z: test accuracy: 0.2518363064008394

--rotation_angle 90 --rotation_axis z: test accuracy: 0.2402938090241343

seg: (num_points=1000)

--rotation_angle 30 --rotation_axis z: test accuracy: 0.6605607779578606

--rotation_angle 45 --rotation_axis z: test accuracy: 0.540841166936791

--rotation_angle 60 --rotation_axis z: test accuracy: 0.47268233387358183

--rotation_angle 90 --rotation_axis z: test accuracy: 0.37488492706645055

From 0 to 90 degree, both cls and seg acc rates are affected significitly and the larger rotation angles will lead to worse results.

cls:

--num_points 1000: test accuracy: 0.9751243781094527

--num_points 500: test accuracy: 0.7305389221556886

seg:

--num_points 1000: test accuracy: 0.8830558882235529

--num_points 500: test accuracy: 0.8114930139720559