Behavioral state detection of newborns based on facial expression analysis

L. Hazelhoff, Jungong Han, S. Bambang Oetomo, P.H.N. With, de

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

10 Citations (Scopus)
178 Downloads (Pure)


Prematurely born infants are observed at a Neonatal Intensive Care Unit (NICU) for medical treatment. Whereas vital body functions are continuously monitored, their incubator is covered by a blanket for medical reasons. This prevents visual observation of the newborns during most time of the day, while it is known that the facial expression can give valuable information about the presence of discomfort. This prompted the authors to develop a prototype of an automated video survey system for the detection of discomfort in newborn babies by analysis of their facial expression. Since only a reliable and situation-independent system is useful, we focus at robustness against non-ideal viewpoints and lighting conditions. Our proposed algorithm automatically segments the face from the background and localizes the eye, eyebrow and mouth regions. Based upon measurements in these regions, a hierarchical classifier is employed to discriminate between the behavioral states sleep, awake and cry. We have evaluated the described prototype system on recordings of three healthy newborns, and we show that our algorithm operates with approximately 95% accuracy. Small changes in viewpoint and lighting conditions are allowed, but when there is a major reduction in light, or when the viewpoint is far from frontal, the algorithm fails. © 2009 Springer Berlin Heidelberg.
Original languageEnglish
Title of host publicationProceedings of the 11th International Conference on Advanced Concepts for Intelligent Vision Systems(ACIVS 2009) 28 September - 2 October 2009, Bordeaux
EditorsJ. Blanc-Talon, W. Philips, D. Popescu, P. Scheunders
Place of PublicationBerlin
ISBN (Print)978-3-642-04696-4
Publication statusPublished - 2009

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743


Dive into the research topics of 'Behavioral state detection of newborns based on facial expression analysis'. Together they form a unique fingerprint.

Cite this