Video-based discomfort detection for infants using a Constrained Local Model

Automatic discomfort detection for infants is important in health care, since infants have no ability to express their discomfort. In this paper, we propose an automatic system for detecting and monitoring discomfort of infants based on video analysis. The system is based on supervised learning and classifies previously unseen infants from the testing set in a fully automated way. Our system consists of face detection and discomfort detection. For each frame, we first detect a face area by using a combination of a skin-color detector and a ViolaJones face detector, and then fit a face shape to the detected face area by using a Constrained Local Model (CLM). After that, we extract expression features by using Elongated Local Binary Pattern (ELBP) on a similarity-normalized appearance (SAPP), and classify expression features with a Support Vector Machine (SVM) for discomfort detection. The key contribution of our system is that it is infant independent and requires no prior knowledge about previously unseen infants. The face detector of the system has an accuracy of 81.5%. The system detects discomfort with an accuracy of 84.3%, a sensitivity of 82.4%, and specificity of 84.9% on the testing set containing videos of 11 infants.