Micro-expression detection in long videos using optical flow and recurrent neural networks

Michiel Verburg, Vlado Menkovski

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)
1 Downloads (Pure)

Abstract

Facial micro-expressions are subtle and involuntary expressions that can reveal concealed emotions. Microexpressions are an invaluable source of information in application domains such as lie detection, mental health, sentiment analysis and more. One of the biggest challenges in this field of research is the small amount of available spontaneous micro-expression data. However, spontaneous data collection is burdened by time-consuming and expensive annotation. Hence, methods are needed which can reduce the amount of data that annotators have to review. This paper presents a novel micro-expression spotting method using a recurrent neural network (RNN) on optical flow features. We extract Histogram of Oriented Optical Flow (HOOF) features to encode the temporal changes in selected face regions. Finally, the RNN spots short intervals which are likely to contain occurrences of relevant facial micro-movements. The proposed method is evaluated on the SAMM database. Any chance of subject bias is eliminated by training the RNN using Leave-One-Subject-Out cross-validation. Comparing the spotted intervals with the labeled data shows that the method produced 1569 false positives while obtaining a recall of 0.4654. The initial results show that the proposed method would reduce the video length by a factor of 3.5, while still retaining almost half of the relevant micro-movements. Lastly, as the model gets more data, it becomes better at detecting intervals, which the proposed method suitable for supporting the annotation process.

Original languageEnglish
Title of host publication14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019
Place of PublicationPiscataway
PublisherInstitute of Electrical and Electronics Engineers
Number of pages6
ISBN (Electronic)978-1-7281-0089-0
DOIs
Publication statusPublished - 1 May 2019
Event14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019 - Lille, France
Duration: 14 May 201918 May 2019

Conference

Conference14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019
CountryFrance
CityLille
Period14/05/1918/05/19

Fingerprint Dive into the research topics of 'Micro-expression detection in long videos using optical flow and recurrent neural networks'. Together they form a unique fingerprint.

  • Cite this

    Verburg, M., & Menkovski, V. (2019). Micro-expression detection in long videos using optical flow and recurrent neural networks. In 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019 [8756588] Piscataway: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/FG.2019.8756588