Health/Physical Condition Monitoring (for human & pigs)

Fall 2018 - present, Postdoctoral Research Associate, Carnegie Mellon University

Occupant Behavior Digression Detection for Alzheimer (in progress: data collection)

Sleep Apnea Study (in progress: data collection)

Pig in Distress

Millions of piglets are crushed by their mother per year in US in the first three days of their lives. To be able to detect this situation and alert the staff to save the piglets in distress, we rely on structural vibration caused by pig activities.


Publications: DATA2018

Individual Perspiration Monitoring

Individual perspiration condition indicates the physical condition of the person. Such information can be used for fitness assessment, patient/elderly care, etc. Compared to prior works, our low-cost sensor can achieve perspiration sensing non-intrusively.
We designed our textile sensor using conductive threads with a stable 3D cotton structure. The cotton braids absorb the sweat and the system measures the conductivity changes between these conductive threads caused by sweat quantity changes. We calibrate the sensor by mapping the measured sensor output voltage and the sweat quantity on the sensor. With this calibration, we were able to conduct on-body sweat sensing and profile individual perspiration quantity and perceived levels using this sensor.


Publications: CPD2018

Non-Intrusive Gait Analysis

Gait analysis reveals individual physical conditions. For example, walking balance is an important indicator of fall risks. Compared to wearable and camera-based methods, we provide a non-intrusive sensing approach through structural vibration sensing, which makes non-clinical monitoring in home scenarios more feasible.


Publications: IMAC2018

Heartbeat Monitoring

Monitoring people's physical conditions non-intrusively and uniquitously are important for various smart buildings. For vehicles, especially autonamous vehicles in the near future, be able to monitor the driver and passenger's physical condition such as heartrate continously allows safer driving.


Publications: HotMobile2018

Indoor Human Monitoring through Structual Vibration Sensing

Fall 2013 - Spring 2018, PhD Student, Carnegie Mellon University

Indoor human information enables various of smart building applications. Pedestrian identity, location and activities can be used to further provide personalized services in the building, and the statistics of this data can be used to improve marketing based on behavior patterns.
Existing sensing approaches, including vision-, sound-, and PIR-based methods have their limitations on sensing condition, e.g., light-of-sight, and raise privacy concerns sometimes. On the other hand, ambient structural vibration signal contains various information of occupant information and activity. Different human activities (walking, running, jumping, moving appliances, cooking, etc.) induce the structure to vibrate in their specific ways.
We present a structural vibration based sensing method that obtains and analyzes this vibrations. Our system can infer the identity, location, traffic count of pedestrians, as well as their activities in the envrionment.

Indoor Human Identification

People walk differently, and the locomotion achieved through their limb movement is referred as gait. Gait has been used as an efficient biometric to identify people. Unique gait results in unique impact on the floor, which induce the floor to vibrate in a way that can be distinguished from others. Therefore, by monitoring and analyzing the floor vibration induced by people's footstep, our system can identify the pedestrian in the indoor environment.


Figure above shows the detected footstep-induced floor vibration signal of two different persons. We can observe that their way of striking the floor are different, especially the angle of the contacting point and the center of gravity. These differences lead to a clear difference in their footstep signals in both time and frequency domain.

Publications: HotMobile2015,Ubicomp2017

Human-Environment Interaction Tracking

We investigate two different level of human-environment interaction here: building level and object level. The building level tracking is referring to step-level pedestrian tracking. The object level tracking allows the system to turn any object surface into a touchscreen.
From the type of interaction perspective, there are mainly two types of interactions: impact induced impulse-like signal (e.g., footstep, finger tapping) and friction induced slip-pulse-like signal (e.g., drag a chair, finger swiping).


Our algorithm treats these two types of waves differently based on their properties, and conduct interaction location estimation through different approaches as shown in the figure above.

Publications: IPSN2017,JSV2017

Occupancy Estimation

We approach indoor occupancy through two approaches: 1) tracking the direction and the event of entering/leaving room, 2) estimating the occupant traffic through a sensing area. The intuition is shown in figure below. The system first detects the footstep induced vibration based on the signal energy change and a Gaussian model built from the noise signals. The closer the person's footstep to the sensor, the higher footstep signal energy.


Therefore, we can estimate the step when the person passing by the sensor -- and with multiple sensors, we can estimate the walking direction based on the timing of the person passing by each sensor. After tracking each individual's walking direction, the system estimates the overall occupancy level of different areas in a building.

Publications: SPIE2014,SPIE2016

Multiple People Sensing

Summer 2016, Research Intern, Technicolor Research, Palo Alto, CA

The intuition to sense multiple human within the same sensing area through floor vibration is to utilize the randomness of the human behavior caused signal offset. Human gait has eight phases (from initial contact to terminal swing). For a specific moment, even for people walk side by side, they may not be in the same phase of their gait cycle, therefore, their feet contact the floor at different moments, as shown in the figure below.


The energy of a foot strike induced vibration is often concentrated at the signal onset part, as shown in the spectragram above. Therefore, the asynchony of people's footstep can be detected. In addition, each sensor receives a superposition of multiple signals from different sources with different decay and delay. Therefore, our approach to learning information when multiple people's footstep signal are mixed is through a combination of 1) the comparison of multiple sensor's signals and 2) the analysis of the signal detected by individual sensors.

Publications: SenSys2016(poster),SPIE2016

Building Energy Saving through Human-in-the-Loop Sensing Systems

Summer 2012 - Spring 2013

Social Games (demo paper)

Fall 2012 - Spring 2013, PhD student, Carnegie Mellon University

Saving building power consumption is important for reducing the cost of the building. This project is aiming at exploring how the incentive in social aspect would effect people's behavior in energy saving, especially in commercial buildings. The ultimate goal of the project is to save energy consumption in commercial buildings through social games.
We used the FireFly smartmeter to monitoring personal and shared sources energy consumption. The power usage of each monitored device is sent to the server. Smart phones are utilized in providing user status sensing and convinient user interface for the gaming system. The game rules are designed to encourage users to turn off devices that are sensed as in the status of wasting power.

iCEnergy (demo paper)

Summer 2012, Research Assistant, Tsinghua University

iCEnergy is a vision-based mobile information interface that provides power monitoring using augmented reality. The system overlays an interactive "Energy Cloud" over corresponding devices in order to illustrate information about the physical environment.

The approach aims to provide users with a comfortable interaction experience through it's intuitive information display. By providing a more intuitive display, the system improves the awareness of the energy usage in the building, hence help incent the user to turn off high power consumption devices when they are not in use.

Headio: Mobile Device Orientation Acquisition

September 2012 - March 2013 , PhD student, Carnegie Mellon University

Various of mobile application, especially location/navigation related, requires accurate orientation information. Headio is a novel approach to provide reliable orientation information for mobile devices in indoor environments using ceiling images.
Most of the time, when people use their mobile phone, one of the camera might capture the direction of the ceiling in an indoor environment. Headio aggregates pictures of the ceiling of an indoor environment and applies computer vision based pattern matching techniques to extract the orientation of the pattern lines on the ceiling.
Then the system intergrates the building orientation obtained from online map, ceiling pattern orientation obtained from the photos, as well as digital compass readings, to calculate accurate device orientation.

Physical Arrangement Detection of Networked Devices through Ambient-Sound Awareness

Spring 2011, Exchange Scholar, Carnegie Mellon University

PANDAA (Physical Arrangement Detection of Networked Devices through Ambient-Sound Awareness) is an indoor acoustic sensing system that features automatic self-localization of networked devices through ambient sound-awareness. There is no pre-installed infrastructure required. And the system is readily applicable for commercial off-the-shelf mobile devices.

The system utilizes multiple impulse like sound sources, such as coughing, door closing, etc., as well as multiple mobile devices in the environment, to obtain relative location of all the devices. The system achieves considerable accuracy and the demo won the Best Demo Award at Ubicomp 2011.

SugarTrail: Indoor Navigation

Fall 2011, Exchange Scholar, Carnegie Mellon University

SensorFly is a low cost aerial embedded sensor networking platform, it is helicopter formed light weighted robot that can fly around sensing environment with various sensors. It is designed to be able to swarm and communicate with each other during tasks. It can be used in emergency situations such as fire and earthquake for survivor seeking and navigation assistance.

We developed a novel localization and navigation system on SensorFly platform named SugarTrail. SugarTrail  uses integrated signatures consisted of distance information from round-trip time of flights (RToF) of radio signals and 3D compass readings to location and eventually navigate moving devices. We deployed the system in supermarket and laboratory for localization data collecting.

Securitas: Mobile Authentication through Hand Geometry

Summer 2013, Interim Engineering Intern, Qualcomm

The security of various mobile smart devices is an important aspect of the system especially with the booming IoT services. Different bio metrics have been explored to allow high accuracy device authentication. Only very limited sensing methods (fingerprint sensor) have been deployed on current mobile phones so far.
We present a method of ubiquitous user identification on mobile devices. By utilizing the NIR-RGB camera and hand geometry characteristics, we provide an alternative identification method on smartphones. The NIR-RGB camera allows robust human skin detection through images, therefore enables the extraction of accurate hand geometry under various sensing conditions. We have achived over 94% accuracy in the identification test.

Nataero: Indoor Assets Tracking

Summer 2013 and Summer 2014, Interim Engineering Intern, Qualcomm

For large scaled inventory, tracking large amount of devices and items would allow companies and research labs save the cost for purchasing and replacing devices.
The challenge including two aspects: 1) how to design low cost low power tags that can be easily attached to various of devices, and 2) how to localize these tracking tags accurately to allow fast tracking.
My task in the project include 1) the BLE tagperformance analysis, 2) mobile application development for the system, and 3) BLE tag energy consumption profiling.

vLoc: Indoor Localization

Summer 2011, Research Intern, Microsoft Research Asia

Indoor localization of mobile devices enables applications in scenarios such as museum, supermarket, office, etc., where people can obtain location-based information from their devices, such as exhibits information, navigation through aisles, and personalized office.

vLoc is an indoor localization system which can be used for in-building office and underground shopping mall. It uses sensors on mobile phone to capture Wi-Fi signal characteristics and user motion, therefore generates the virtual map of the indoor environment. With the virtual map, users with mobile devices can be located.