The MatchNMingle dataset: a novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates

Laura Cabrera-Quiros (Corresponding author), Andrew Demetriou (Corresponding author), Ekin Gedik, Leander van der Meij, Hayley Hung

Research output: Contribution to journalArticleAcademicpeer-review

40 Citations (SciVal)
393 Downloads (Pure)

Abstract

We present MatchNMingle, a novel multimodal/multisensor dataset for the analysis of free-standing conversational groups and speed-dates in-the-wild. MatchNMingle leverages the use of wearable devices and overhead cameras to record social interactions of 92 people during real-life speed-dates, followed by a cocktail party. To our knowledge, MatchNMingle has the largest number of participants, longest recording time and largest set of manual annotations for social actions available in this context in a real-life scenario. It consists of 2 hours of data from wearable acceleration, binary proximity, video, audio, personality surveys, frontal pictures and speed-date responses. Participants' positions and group formations were manually annotated; as were social actions (eg. speaking, hand gesture) for 30 minutes at 20fps making it the first dataset to incorporate the annotation of such cues in this context. We present an empirical analysis of the performance of crowdsourcing workers against trained annotators in simple and complex annotation tasks, founding that although efficient for simple tasks, using crowdsourcing workers for more complex tasks like social action annotation led to additional overhead and poor inter-annotator agreement compared to trained annotators (differences up to 0.4 in Fleiss' Kappa coefficients). We also provide example experiments of how MatchNMingle can be used.
Original languageEnglish
Article number8395003
Pages (from-to)113-130
Number of pages18
JournalIEEE Transactions on Affective Computing
Volume12
Issue number1
DOIs
Publication statusPublished - 1 Jan 2021

Funding

Thanks to Sebastian Deuten, Veronica de Groene, Shari Molawi, Julia Emmer, Phuong Khanh Vu, Marina Tulin, Clara De Inocencio, Maria I. Rinderu, Jordy Jouby and Catherine Mohlo for their help collecting this dataset; Maarten Jonker and Leon Harmsen, managers of Il Cafe; and also our anonymous participants. This paper was partially funded by the Dutch national program COMMIT, the Instituto Tec-nológico de Costa Rica and the Netherlands Organization for Scientific Research (NWO) under project number 639.022.606. Laura Cabrera-Quiros and Andrew Demetriou are contributed equally.

Keywords

  • Acceleration
  • Cameras
  • cameras
  • Computers
  • Crowdsourcing
  • f-formation
  • Manuals
  • mingle
  • Multimodal dataset
  • personality traits
  • Sensors
  • Speed-dates
  • Task analysis
  • wearable acceleration

Fingerprint

Dive into the research topics of 'The MatchNMingle dataset: a novel multi-sensor resource for the analysis of social interactions and group dynamics in-the-wild during free-standing conversations and speed dates'. Together they form a unique fingerprint.

Cite this