Cai's team delivered a parallel neural network system to NASA Langley in July, 2003.
Visual Intelligence Studio

Visual Intelligence Studio (VIS)

MISSION

The mission of VIS is to develop perceptive computing technology that is designed to capture complex patterns, relationships and exceptions in a data set. It is a way to summarize seemingly disjoint data into significant parts and pass the summary information to decision entities. It is a new field that combines cognitive pattern recognition, machine learning, visualization and data mining. The strength of VIS includes onboard neural vision, texture analysis, and scientific visualization. The applications include new products for intelligent transportation systems, inspections, medical image processing and remote sensing.

FACILITY

VIS is equiped with the state-of-the-art development platforms. For example, the resolution independent human modeling software, Zero Instruction Set Computing (ZISC) development system, the visual appearance modeling system, high resolution network cameras, FEMLAB multiphysics modeling system and color calibration systems.

PROJECTS

1. Vision in Vehicles (VIV)

This project is a joint R&D with Bombardier Transportation and Transportation Research Board (TRB), National Academy of Sciences (NAS). So far TRB and Bombarider have committed grants for supporting this project. The new phase will be started from October, 2004 to December 2006. According to a Public Transit Association report, there is a need for $5.3 billion dollars for upgrading transit security systems, including real-time surveillance video and face recognition. Unfortunately, technology for those applications does no exist because of the mobile environment and complexity of targets. This project includes two phases: 1) developing a test site at Bombardier's test track for transferring wireless video from a moving vehicle to the control center; 2) develop intelligence algorithms for face recognition, sensor fusion and abnormal event detection. Those algorithms are designed to be seamlessly integrated into the existing control and communication software systems. The academic challenges include: 1) face detection in a moving and light-variation environment, 2) feature representation for normal or abnormal events, 3) human-camera interaction and adaptive display design for maximal vigilance and minimal bandwidth both for human attention and ad hoc wireless networks.

Human Detection from Video

2. Spatiotemporal Data Mining (SDM)

The project is sponsored by NASA, starting October 1, 2004 to September 30, 2006 with a grant of $400,000 for CMU. Tracking and modeling spatiotemporal dynamics of ocean objects are essential to NASA and NOAA missions in oceanographic studies, such as monitoring and predicting harmful algal blooms along the coastline, or river-based plume discharged to the open ocean. Despite of successful case studies in detecting ocean objects based on satellite images, most of work has been done manually, such as area marking and measurement, which is expensive and time consuming. We still lack general data mining algorithms for automatically monitoring the movement of an ocean object and forecasting the object movement. Besides, most of current studies are based on a single database rather than multiple data resources. In this project, we develop a spatiotemporal data mining system for following objectives: 1) tracking the movement of ocean objects that have been identified; 2) discovering the correlations between the object attributes and satellite readings from multiple databases; 3) predicating the movement of identified objects. This generalized spatiotemporal data mining tool enables monitoring and modeling for multiple oceanographic objects, such as plume and harmful algal blooms. This may also be applied to other spatiotemporal problems, such as monitoring dust storms. We use SeaWiFS database as our main source. Meanwhile, we will explore the use of other remote sensing databases such as MODIS. The technology would be based on our lab prototypes of multi-sensor data mining framework with the entrance Technical Readiness Level 4. The project deliverable would reach TRL 5 to 6. The total time for this project is for two years. The Co-PI Dr. Richard P. Stumpf, Oceanographer from NOAA will specify the requirements for the data mining tool and validate the product with field data. Dr. Han-Shou Liu, Geophysicist of GSFC, will support computational models for data mining. Dr. Horace Mitchell, Director of SVS, GSFC will support visualization of data mining results. Dr. Judith Devaney, Group Leader, Scientific Applications and Visualization Group, NIST, will participate the study on high fidelity virtual reality system for spatiotemporal data mining.

3. Interactive Biological Simulation (BioSim)

Understanding and solving biomedical problems requires insight into the complex interactions between the components of biomedical systems by domain and non-domain experts. This is challenging because of the enormous amount of data and knowledge in this domain. Therefore, non-traditional educational tools have been developed such as a biological storytelling system, animations of biomedical processes and concepts, and interactive virtual laboratories. The next-generation problem solving tools need to be more interactive to include users with any background, while remaining sufficiently flexible to target open research problems at any level of abstraction, from the conformational changes of a protein to the interaction of the various biochemical pathways in our body. Here, we present an interactive and visual problem-solving environment for the biomedical domain. We designed a biological world model, in which users can explore biological interactions by role-playing "characters" such as cells and molecules or as an observer in a "shielded vessel", both with the option of networked collaboration between simultaneous users. The system architecture of these "characters" contains four main components: (1) Bio-behavior is modeled using cellular automata. (2) Bio-morphing uses vision-based shape tracking techniques to learn from recordings of real biological dynamics. (3) Bio-sensing is based on molecular principles of recognition to identify objects, environmental conditions and progression in a process. (4) Bio-dynamics implements mathematical models of cell growth and fluid-dynamic properties of biological solutions. The principles are implemented in a simple world model of the human vascular system and a biomedical problem that involves an infection by Neisseria meningitides where the biological characters are white and red blood cells and Neisseria cells. Our case studies show that the Problem Solving Environment can inspire user's strategic, creative and innovative thinking.

Local Preschool Students Played BioSim

4. Image Feature Indexing

People use feature indexing in daily life. For example, we say someone looks like a "Superman." We represent an image with a word. Feature indexing is important to fast image retrieval, compression and reconstruction. Currently, we are working on a research project sponsored by NASA Innovation and Creativity Program for Onboard Lidar Image Feature Indexing. We use the unsupervised machine learning algorithm to cluster the NASA satellite lidar profile data and generate an index dictionary. Then we train the model with the index dictionary with a neural network model. Finally, we use the model for the onboard feature recognition and indexing. We implemented a prototype of the onboard computer with ZISC (Zero Instruction Set Computing) chips so that it takes advantage of intrinsic parallel computing and reconfigurability. We tested a set of 44,000 profiles of NASA satallite lidar images that make up the indexing dictionary. With 64 indices, we reach a high compression rate 99.17% with an acceptable error range. We found the required neurons are equal to the indices. We also compared our method to wavelet algorithm and found that it significantly outperforms the wavelet compression technique.

5. Texture Analyzer

A digital imaging and modeling system is developed for tongue inspection that has been an important diagnostic method of Traditional Chinese Medicine (TCM). The system includes image acquisition, color calibration, image segmentation, feature extraction and classification functions. The preliminary results show that the color and texture features are sensitive to the abnormal tongues. The technology include portable texture imaging system and color calibration software.

Crack Finder

6. Spatiotemporal Cell Motion Profiler (CellTrack)

The cultured cells can live outside of the body for many hours. The time-lapsed digital movies of the live cells are captured for studies of cellular behavior. The patterns of cell movement and cell-to-cell interactions can reveal the biological signatures. The red outline indicates the computer vision trackingoutput. The system can automatically track the cell and generate 2D profiles.

2D Profiler Output

PERSONNEL

Dr. Yang Cai, Director of VIS. Dr. Cai is in charge of the development,project management and research. His area includes computer vision and scientific visualization. www.andrew.cmu.edu/~ycai

Teri Mick, Software Engineer. Ms. Mick is involved NASA sponsored projects and modeling work for Capital Technology, Inc.

Paul Li, Research Programmer. Mr. Li is Electrical Engineering student at CMU. He is working on wireless video streaming and onboard pattern recognition project.

Patty Pun, Research Programmer. Mr. Li is Electrical Engineering student at CMU. He is working on wireless video streaming and microwave modeling.

Daniel Chen, Research Programmer. Mr. Chen is INI student at CMU. He is working on VIV project.

Mark Tomczak, Research Programmer. Mr. Tomczak is Computer Science student at CMU. He worked on the reconfigurable parallel neural network computer for NASA Langley Research Center.

COLLABORATORS

Dr. Judith Klein Seetharaman, Assistant Professor of Biochemistry, University of Pittsburgh Medical Center.

Dr. Bill Eddy, Professor, CALD and Statistics, CMU

Dr. Judith Devaney, Program Manager, NIST

Dr. Juyce Anastasi, Associate Professor, Presby Medical Center, Columbia University.

Dr. Y. Hu, NASA Langley Research Center.

SPONSORS

NASA
Jewish Healthcare Foundation
National Research Council
PITA
Anonymous Foundation

CONTACT

Dr. Yang Cai
Visual Intelligence Studio,
Carnegie Mellon University
700 Technology Dr.
Pittsburgh, PA 15213
ycai@cmu.edu
412-268-1518 (phone)
www.andrew.cmu.edu/~ycai