Companionbot for dialog on Depression: This collaborative research project aims at developing a new class of dialog-based, home robotic healthcare assistants to facilitate a new level of in-home, real-time care to elderly and depressed patients, providing lower total costs and higher quality of life. An emotive, physical avatar called companionbot that possesses the ability to engage humans in a way that is unobstructive and suspends disbelief will be built in this project. The companionbot will be an integration of human language technology, vision, other sensory processing and emotive robotic technology to proactively recognize and dialog with isolated and elderly patients suffering from depression. The companionbot will utilize proactive or companionable dialog based on the context with users suffering from depression. This will require the first multimodal integration of a user model, environment model, and temporal processing with spoken dialog understanding and generation to produce dynamic dialog and emotive interaction, beyond the traditional scripted dialog and emotion. Object recognition, facial expressions recognition, and human activity recognition will augment natural language processing to provide current and historical context important to dynamic dialog. This research is supported by grant IIS-111568 from the National Science Foundation.

Autism Robot Assist: This project aims at to develop and utilize humanoid robots for analyzing   behavior principles of children with ASDs and go beyond simple goals of using robots as a neat   toy to make kids smile or maybe even engage in joint attention. Our goal is to use the robotic   technology as co-robot to interpret behavior, determine if behavior is appropriate, provide   feedback on different behavior, provide adaptation on for behavior therapy progress, and reward   if behavior is improved towards therapeutic goals. We use Nao and Zeno, two commercially   available  humanoid robots and develop the necessary features such as facial expressions   recognition, speech processing, and multi-modal feedback system for automatic interventions   with  children with ASDs. Families who would like to participate in this project, please see find   information here: http://www.engr.du.edu/mmahoor/flyer-ASDRobotAssist.pdf

Search and Rescue: The earthquake and tsunami that devastated parts of Japan is a terrible reminder of the power and danger of natural disasters. While we respectfully mourn the tragic loss of life and the hardships that the living are experiencing and will continue to experience for months and years to come, there are many opportunities to learn, to contribute, and to improve on current response methods. In this RAPID project, we aim to assist Japanese colleagues in the robotic response to the disaster in the form of vision improvements to the CRAWLER (Cylindrical Robot for Autonomous Walking and Lifting during Emergency Response) robot for core-bored search and void inspection. More information about this project can be found at:  http://www.engr.du.edu/mmahoor/Rapid-Project.htm

This research is supported by grant CNS-1138674 from the National Science Foundation.

 Unmanned Systems: The main objective and ultimate goal of this MRI development proposal is to enable research into the foundations of the next generation of autonomous Unmanned Aerial Systems (UAS) that will be used for a wide spectrum of civilian applications like earth science, environment/pollution monitoring, agriculture, land management, civil infrastructure health management, public security, fire control, emergency response, etc. We are developing one multi-functional, intelligent instrument composed of two tightly coupled components: a prototype, novel, lightweight (<150 Kg Maximum Take Off Weight) unmanned helicopter and a general-purpose landing platform that will also serve as a refueling/recharging station and data relay/repository. The need for such an instrument is clear since current airborne payload technology is limited and helicopters of this scale suffer from extremely limited endurance. This tightly coupled instrument may be viewed as an enhanced, intelligent mobile sensor serving as an autonomous node in a distributed intelligence wireless network, or a wireless sparse mobile network of robot teams.

Spontaneous Facial Emotion Recognition: In this research project, our focus is on developing automated computer vision techniques for measuring/recognizing spontaneous facial expressions and FACS action units and map onto expert human coding. We apply developed automated techniques to analyze emotion expressions of children with autism. Autism is a severe early childhood developmental disorder that is characterized by deficits in cognitive performance as well as progressive qualitative impairments in social interactions. In the absence of a reliable biological marker, diagnosis of this devastating disorder is still based on its clinical manifestations. One of the clinical features of autism is difficulties in appropriately creating facial expressions that reflect child’s emotions. This research is supported by grant BCS-1052781 from the National Science Foundation.

Facial Features Extraction: Facial feature extraction is an important step in face recognition and is defined as the process of locating specific regions, points, landmarks, or curves/contours in a given 2-D image or a 3-D range image. Active Shape Model (ASM) is a statistical approach for shape modeling and 2-D feature extraction using a parameterized statistical shape model obtained from a training set of manually labeled features. In my research work, I improved the ASM method for extraction of facial features. The core of the enhancement relies on incorporating color information to present the local structure of the feature points. In addition, we developed an algorithm for labeling facial features (i.e., the two corners of the eyes and the tip of the nose), in 3-D range images. 

Multi-Modal Ear and Face Recognition: For more than three decades, researchers have worked in the area of face recognition. Despite these efforts, face recognition is not ready yet for real world applications. Ear biometric is a relatively new area of research. There have been few studies conducted using 2-D data (image intensity) and 3-D shape data. Because of the lack of robustness of a single biometric, multimodal biometric have caught the attention of researcher in the area of computer vision. There are several motivations for a multi-modal ear and face biometric. First, the ear and face data can be captured using regular cameras. Second, the data collection for face and ear is nonintrusive. Third, the ear and face are physically close to each other and most of the time acquiring data for ear (face) encounters face (ear) too. Most of the time, these two bio-markers exist in an image or video captured from humans’ head and both are available to a biometric system. Thus, a multi-modal face and ear recognition system is more feasible than a multi-modal face and fingerprint recognition system. Based on the above discussion, we present a multi-modal ear and face biometric system.  

Graph Cut Optimization and Its Application in Image Blending:: In the area of computer vision, optimization is an important issue. One of the new developed techniques that caught my attention recently is “Graph-Cut”. I worked on using this new optimization technique in applications such as image blending or image segmentation. I presented a novel approach for combining a set of registered images into a composite mosaic with no visible seams and minimal texture distortion. Pair-wise image blending was performed by means of watershed segmentation and Graph-Cut optimization. This approach is fast because searching the optimal intersection between the images was restricted over a small set of watershed segments, instead of optimizing over the entire set of pixels in the intersection zone. The solution is found efficiently via Graph-Cut, using a given photometric criterion.