Dual-camera facial landmark tracking for clinical infant monitoring

This paper presents a new algorithm for dual-camera facial landmark tracking, intended for clinical infant monitoring. For landmark tracking of in-the-wild faces, state-of-the-art solutions use deep networks for face detection and landmark localization. For clinical infant monitoring, however, there is currently not enough data to reliably train such networks. The proposed algorithm overcomes this by incorporating 3D head tracking, to supplement face detection and to enable a refinement version of landmark localization that is more easily learned from less data. The paper reports experimental results on real-life video of infants in bed in hospital (in visual light or near-infrared light), showing good landmark tracking behavior. For challenging sequences without severe occlusions, the mean tracking error, in terms of the RMS error measured for a subset of landmarks and divided by the maximum eye-philtrum distance, remains below 20%. The proposed algorithm is used successfully in an infant monitoring system under development.