My final project for the course 15-869: Visual Computing Systems
The CMU Smart Headlight is a reactive visual system developed at Carnegie Mellon university under the leadership of prof. Srinivasa Narasimhan of Robotics Institute. I intend to implement a similar high speed image processing/ analysis pipeline using darkroom which is a language for describing hardware image processing pipelines embedded in Terra.
As per the proposed schedule, I have completed the tasks plan for the first two weeks with the exception of adding, dilation on the binarized image. Meanwhile I got the Darkroom up and running on the Tegra K1 development board which wasn’t a task listed in the checklist for week1 and week2.
I have generated a range of input images with a number of illuminated circles with different intensity and sizes to simulate varying illumination as well as varied number of particles. And I will make a couple of uncompressed video files by drawing particles of random sizes (offcourse bounded with in a range).
As planned, I'll deliver the darkroom implementation of the CMU smartheadlight pipeline for tracking and dislluminating/illuminating particles. The demo test case will be pingball ball tracking and illumination. I will try my level best to demo using live inputs from the GS3 Near Infrared Camera as long as there are no logistical issues that cannot be resolved before December 1st. Also I intend to report the end to end latency figures on Tegra K1, because this pipeline has never been ported onto an ARM based platform.
The current status of the project is that, I'm facing issues in integrating the darkroom generated code to the application written in C/C++. The darkroom kernels, and pipeline as a fused kernel works well as a standalone entity with the aid of lua jit compiler. I am aware of the fact that terra lets me compile stuff into objects which can be linked to my application written in C/C++. I've sought help from James. So I'm setting a checkpoint 2.5 to achieve this by Friday. I will be able to profile my code only if I surpass this stage. And this is equally important for me to easily generate the disassembly to analyze how well darkroom leverages NEON and SSE4/AVX capabilities of the host CPU.