A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video

Muhammad Awais; Xi Long; Bin Yin; Saadullah Farooq Abbasi; Saeed Akbarzadeh; Chunmei  Lu; Xinhua Wang; Laishuan Wang; Jiong Zhang; Jeroen Dudink; W. Chen

doi:10.1109/JBHI.2021.3073632

A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video

Muhammad Awais
, Xi Long (Corresponding author)
, Bin Yin
, Saadullah Farooq Abbasi
, Saeed Akbarzadeh
, Chunmei Lu (Corresponding author)
, Xinhua Wang
, Laishuan Wang
, Jiong Zhang
, Jeroen Dudink
, W. Chen (Corresponding author)

Onderzoeksoutput: Bijdrage aan tijdschrift › Tijdschriftartikel › Academic › peer review

52 Citaten (Scopus)

587 Downloads (Pure)

Samenvatting

Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

Originele taal-2	Engels
Artikelnummer	9405399
Pagina's (van-tot)	1441-1449
Aantal pagina's	9
Tijdschrift	IEEE Journal of Biomedical and Health Informatics
Volume	25
Nummer van het tijdschrift	5
Vroegere onlinedatum	15 apr. 2021
DOI's	https://doi.org/10.1109/JBHI.2021.3073632
Status	Gepubliceerd - mei 2021

Financiering

dateofcurrentversionMay11,2021.ThisworkwassupportedinpartApril2,2021;acceptedApril6,2021.DateofpublicationApril15,2021; Swhichisassociatedwithreducedresponsivenesstoexternal bytheShanghaiMunicipalScienceandTechnologyMajorProjectunder stimuli [1], [2]. According to research on human development Grant 2017SHZDZX01, and in part by the National Key R&D Program in early life, sleep is an essential factor for the development of ofChinaunderGrant2017YFE0112000.(Correspondingauthors:Wei the nervous system in infants [3], [4]. Newborn babies usually MuhammadAwais,SaadullahFarooqAbbasi,andSaeedAkbarzadehChen;XiLong;andChunmeiLu.) sleep between 16 and 18 hours per day in equispaced periods. are with the Center for Intelligent Medical Electronics, Department of As age increases, sleep changes from an ultradian rhythm to a ElectronicEngineering,SchoolofInformationScienceandTechnology, circadian rhythm [5]. Consistent evidence indicates that sleep fudan.edu.cn;[email protected];[email protected]).Fudan University, Shanghai 200433, China(e-mail: 17110720061@ is vital for the brain development of neonates (in particular for Xi Long is with the Philips Research, 5656 AE Eindhoven, The Nether-preterm infants) and help them in recovering from illness [2], [6]. lands, and also with the Department of Electrical Engineering, Eind- Further, the reliable measures for the tracking and assessment (e-mail:[email protected]).hovenUniversityofTechnology,5612AZEindhoven,TheNetherlands of wake-sleep patterns, over multiple nights could potentially Bin Yin is with the Connected Care and Personal provide an indication of neonatal development over time [7], Department, Philips Research, Shanghai 200032, China [8], [9], [10].

Financiers
National Key Research and Development Program of China

Duurzame ontwikkelingsdoelstellingen van de VN

Deze output draagt bij aan de volgende duurzame ontwikkelingsdoelstelling(en)

SDG 3 – Goede gezondheid en welzijn

Toegang tot document

10.1109/JBHI.2021.3073632

published versionDefinitieve gepubliceerde versie, 2,78 MBLicentie: TAVERNE

Andere bestanden en links

Link naar publicatie in Scopus

Perinatal Medicine
van der Hout-van der Jagt, B. (Content manager) & Delvaux, E. (Content manager)
Impact: Research Topic/Theme (at group level)
Sleep Medicine
van Gilst, M. (Content manager) & van der Hout-van der Jagt, B. (Content manager)
Impact: Research Topic/Theme (at group level)

Citeer dit

@article{79699cbcaffa45378b3942ce0344062d,

title = "A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video",

abstract = "Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2\% and an F1-score 0.93 ± 0.3.",

keywords = "Brain modeling, Electroencephalography, Hospitals, Monitoring, Neonatal sleep monitoring, Pediatrics, Sleep, Support vector machines, deep convolutional neural network, facial expression, support vector machine, video and image analysis",

author = "Muhammad Awais and Xi Long and Bin Yin and Abbasi, \{Saadullah Farooq\} and Saeed Akbarzadeh and Chunmei Lu and Xinhua Wang and Laishuan Wang and Jiong Zhang and Jeroen Dudink and W. Chen",

year = "2021",

month = may,

doi = "10.1109/JBHI.2021.3073632",

language = "English",

volume = "25",

pages = "1441--1449",

journal = "IEEE Journal of Biomedical and Health Informatics",

issn = "2168-2194",

publisher = "Institute of Electrical and Electronics Engineers",

number = "5",

}

A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video. / Awais, Muhammad; Long, Xi (Corresponding author); Yin, Bin et al.
In: IEEE Journal of Biomedical and Health Informatics, Vol. 25, Nr. 5, 9405399, 05.2021, blz. 1441-1449.

Onderzoeksoutput: Bijdrage aan tijdschrift › Tijdschriftartikel › Academic › peer review

TY - JOUR

T1 - A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video

AU - Awais, Muhammad

AU - Long, Xi

AU - Yin, Bin

AU - Abbasi, Saadullah Farooq

AU - Akbarzadeh, Saeed

AU - Lu, Chunmei

AU - Wang, Xinhua

AU - Wang, Laishuan

AU - Zhang, Jiong

AU - Dudink, Jeroen

AU - Chen, W.

PY - 2021/5

Y1 - 2021/5

N2 - Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

AB - Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

KW - Brain modeling

KW - Electroencephalography

KW - Hospitals

KW - Monitoring

KW - Neonatal sleep monitoring

KW - Pediatrics

KW - Sleep

KW - Support vector machines

KW - deep convolutional neural network

KW - facial expression

KW - support vector machine

KW - video and image analysis

UR - https://www.scopus.com/pages/publications/85104605582

U2 - 10.1109/JBHI.2021.3073632

DO - 10.1109/JBHI.2021.3073632

M3 - Article

C2 - 33857007

SN - 2168-2194

VL - 25

SP - 1441

EP - 1449

JO - IEEE Journal of Biomedical and Health Informatics

JF - IEEE Journal of Biomedical and Health Informatics

IS - 5

M1 - 9405399

ER -

A hybrid DCNN-SVM model for classifying neonatal sleep and wake states based on facial expressions in video

Samenvatting

Financiering

Duurzame ontwikkelingsdoelstellingen van de VN

Toegang tot document

Andere bestanden en links

Vingerafdruk

Impact

Perinatal Medicine

Sleep Medicine

Citeer dit