Spontaneous Facial Emotion Recognition: In this research project, our focus is on developing automated computer vision techniques for measuring/recognizing spontaneous facial expressions and FACS action units and map onto expert human coding. We apply developed automated techniques to analyze emotion expressions of children with autism. Autism is a severe early childhood developmental disorder that is characterized by deficits in cognitive performance as well as progressive qualitative impairments in social interactions. In the absence of a reliable biological marker, diagnosis of this devastating disorder is still based on its clinical manifestations. One of the clinical features of autism is difficulties in appropriately creating facial expressions that reflect child’s emotions.

Facial Features Extraction: Facial feature extraction is an important step in face recognition and is defined as the process of locating specific regions, points, landmarks, or curves/contours in a given 2-D image or a 3-D range image. Active Shape Model (ASM) is a statistical approach for shape modeling and 2-D feature extraction using a parameterized statistical shape model obtained from a training set of manually labeled features. In my research work, I improved the ASM method for extraction of facial features. The core of the enhancement relies on incorporating color information to present the local structure of the feature points. In addition, I developed an algorithm for labeling facial features (i.e., the two corners of the eyes and the tip of the nose), in 3-D range images. 

Multi-Modal Ear and Face Recognition: For more than three decades, researchers have worked in the area of face recognition. Despite these efforts, face recognition is not ready yet for real world applications. Ear biometric is a relatively new area of research. There have been few studies conducted using 2-D data (image intensity) and 3-D shape data. Because of the lack of robustness of a single biometric, multimodal biometric have caught the attention of researcher in the area of computer vision. There are several motivations for a multi-modal ear and face biometric. First, the ear and face data can be captured using regular cameras. Second, the data collection for face and ear is nonintrusive. Third, the ear and face are physically close to each other and most of the time acquiring data for ear (face) encounters face (ear) too. Most of the time, these two bio-markers exist in an image or video captured from humans’ head and both are available to a biometric system. Thus, a multi-modal face and ear recognition system is more feasible than a multi-modal face and fingerprint recognition system. Based on the above discussion, we present a multi-modal ear and face biometric system.