Applying Delaunay Triangulation Augmentation for Deep Learning Facial Expression Generation and Recognition

Hristo Valev, Alessio Gallucci, Tim Leufkens, Joyce Westerink, Corina Sas

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Generating and recognizing facial expressions has numerous applications, however, those are limited by the scarcity of datasets containing labeled nuanced expressions. In this paper, we describe the use of Delaunay triangulation combined with simple morphing techniques to blend images of faces, which allows us to create and automatically label facial expressions portraying controllable intensities of emotion. We have applied this approach on the RafD dataset consisting of 67 participants and 8 categorical emotions and evaluated the augmentation in a facial expression generation and recognition tasks using deep learning models. For the generation task, we used a deconvolution neural network which learns to encode the input images in a high-dimensional feature space and generate realistic expressions at varying intensities. The augmentation significantly improves the quality of images compared to previous comparable experiments and it allows to create images with a higher resolution. For the recognition task, we evaluated pre-trained Densenet121 and Resnet50 networks with either the original or augmented dataset. Our results indicate that the augmentation alone has a similar or better performance compared to the original. Implications of this method and its role in improving existing facial expression generation and recognition approaches are discussed.

Original languageEnglish
Title of host publicationPattern Recognition ICPR International Workshops and Challenges
Subtitle of host publicationVirtual Event, January 10–15, 2021, Proceedings, Part III
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
Place of PublicationCham
PublisherSpringer
Pages730-740
Number of pages11
ISBN (Electronic)978-3-030-68796-0
ISBN (Print)978-3-030-68795-3
DOIs
Publication statusPublished - 2021
Event25th International Conference on Pattern Recognition Workshops, ICPR 2020 - Milan, Italy
Duration: 10 Jan 202115 Jan 2021
Conference number: 25
https://www.micc.unifi.it/icpr2020/

Publication series

NameLecture Notes in Computer Science (LNCS)
Volume12633
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349
Name Image Processing, Computer Vision, Pattern Recognition, and Graphics (LNIP)
Volume12663

Conference

Conference25th International Conference on Pattern Recognition Workshops, ICPR 2020
Abbreviated titleICPR 2020
Country/TerritoryItaly
CityMilan
Period10/01/2115/01/21
Internet address

Keywords

  • Augmentation
  • Deep learning
  • Emotions
  • Facial expressions

Fingerprint

Dive into the research topics of 'Applying Delaunay Triangulation Augmentation for Deep Learning Facial Expression Generation and Recognition'. Together they form a unique fingerprint.

Cite this