Camera-based discomfort detection using multi-channel attention 3D-CNN for hospitalized infants

Yue Sun, Jingjing Hu, Wenjin Wang (Corresponding author), Min He, Peter H.N. de With

Research output: Contribution to journalArticleAcademicpeer-review

8 Citations (Scopus)
62 Downloads (Pure)


Background: Detecting discomfort in infants is an important topic for their well-being and development. In this paper, we present an automatic and continuous video-based system for monitoring and detecting discomfort in infants. Methods: The proposed system employs a novel and efficient 3D convolutional neural network (CNN), which achieves an end-to-end solution without the conventional face detection and tracking steps. In the scheme of this study, we thoroughly investigate the video characteristics (e.g., intensity images and motion images) and CNN architectures (e.g., 2D and 3D) for infant discomfort detection. The realized improvements of the 3D-CNN are based on capturing both the motion and the facial expression information of the infants. Results: The performance of the system is assessed using videos recorded from 24 hospitalized infants by visualizing receiver operating characteristic (ROC) curves and measuring the values of area under the ROC curve (AUC). Additional performance metrics (labeling accuracy) are also calculated. Experimental results show that the proposed system achieves an AUC of 0.99, while the overall labeling accuracy is 0.98. Conclusions: These results confirms the robustness by using the 3D-CNN for infant discomfort monitoring and capturing both motion and facial expressions simultaneously.

Original languageEnglish
Pages (from-to)3059-3069
Number of pages11
JournalQuantitative Imaging in Medicine and Surgery
Issue number7
Publication statusPublished - Jul 2021


  • 3D convolutional neural network (3D-CNN)
  • Discomfort detection
  • Infant discomfort
  • Video health monitoring


Dive into the research topics of 'Camera-based discomfort detection using multi-channel attention 3D-CNN for hospitalized infants'. Together they form a unique fingerprint.

Cite this