Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection

Laura Cabrera-Quiros; Ekin Gedik; Hayley Hung

Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection

Laura Cabrera-Quiros
, Ekin Gedik
, Hayley Hung

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

29 Downloads (Pure)

Abstract

This paper presents the algorithms that task organisers deployed for the automatic Human Behaviour Analysis (HBA) task of the MediaEval 2018. HBA task aims to investigate alternate modalities of video and body-worn acceleration for the detection of speaking status. For unimodal estimation from acceleration, a transfer learning approach, Transductive Parameter Transfer (TPT), which is shown to perform satisfactorily in a similar setting[4] is employed. For the estimation from the video modality, bags of Dense Trajectories were used in a multiple instance learning approach (MILES) [2]. Finally, late fusion is used for combining the outputs from both modalities. The multi-modal approach resulted in a mean AUC of 0.658, outperforming the performance of both single modality approaches. Copyright held by the owner/author(s).

Original language	English
Title of host publication	MediaEval 2018 Multimedia Benchmark Workshop
Subtitle of host publication	Working Notes Proceedings of the MediaEval 2018 Workshop Sophia Antipolis, France, 29-31 October 2018
Editors	Martha Larson, Claire-Hélène Demarty, Piyush Arora
Publisher	CEUR-WS.org
Number of pages	3
Publication status	Published - 1 Jan 2018
Externally published	Yes
Event	2018 MediaEval Workshop - Sophia Antipolis, France Duration: 29 Oct 2018 → 31 Oct 2018

Publication series

Name	CEUR Workshop Proceedings
Volume	2283
ISSN (Print)	1613-0073

Conference

Conference	2018 MediaEval Workshop
Country/Territory	France
City	Sophia Antipolis
Period	29/10/18 → 31/10/18

Funding

This task is partially supported by the Instituto Tecnológico de Costa Rica and the Netherlands Organization for Scientific Research (NWO) under project number 639.022.606.

Access to Document

published versionFinal published version, 1.16 MB

http://ceur-ws.org/Vol-2283/

Cite this

Cabrera-Quiros, L., Gedik, E., & Hung, H. (2018). Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection. In M. Larson, C.-H. Demarty, & P. Arora (Eds.), MediaEval 2018 Multimedia Benchmark Workshop: Working Notes Proceedings of the MediaEval 2018 Workshop Sophia Antipolis, France, 29-31 October 2018 Article 3 (CEUR Workshop Proceedings; Vol. 2283). CEUR-WS.org. http://ceur-ws.org/Vol-2283/

Cabrera-Quiros, Laura ; Gedik, Ekin ; Hung, Hayley. / Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection. MediaEval 2018 Multimedia Benchmark Workshop: Working Notes Proceedings of the MediaEval 2018 Workshop Sophia Antipolis, France, 29-31 October 2018. editor / Martha Larson ; Claire-Hélène Demarty ; Piyush Arora. CEUR-WS.org, 2018. (CEUR Workshop Proceedings).

@inproceedings{7e74996a78eb42c9a3b0646f46ffbd00,

title = "Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection",

abstract = "This paper presents the algorithms that task organisers deployed for the automatic Human Behaviour Analysis (HBA) task of the MediaEval 2018. HBA task aims to investigate alternate modalities of video and body-worn acceleration for the detection of speaking status. For unimodal estimation from acceleration, a transfer learning approach, Transductive Parameter Transfer (TPT), which is shown to perform satisfactorily in a similar setting[4] is employed. For the estimation from the video modality, bags of Dense Trajectories were used in a multiple instance learning approach (MILES) [2]. Finally, late fusion is used for combining the outputs from both modalities. The multi-modal approach resulted in a mean AUC of 0.658, outperforming the performance of both single modality approaches. Copyright held by the owner/author(s).",

author = "Laura Cabrera-Quiros and Ekin Gedik and Hayley Hung",

year = "2018",

month = jan,

day = "1",

language = "English",

series = "CEUR Workshop Proceedings",

publisher = "CEUR-WS.org",

editor = "Martha Larson and Claire-H{\'e}l{\`e}ne Demarty and Piyush Arora",

booktitle = "MediaEval 2018 Multimedia Benchmark Workshop",

note = "2018 MediaEval Workshop ; Conference date: 29-10-2018 Through 31-10-2018",

}

Cabrera-Quiros, L, Gedik, E & Hung, H 2018, Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection. in M Larson, C-H Demarty & P Arora (eds), MediaEval 2018 Multimedia Benchmark Workshop: Working Notes Proceedings of the MediaEval 2018 Workshop Sophia Antipolis, France, 29-31 October 2018., 3, CEUR Workshop Proceedings, vol. 2283, CEUR-WS.org, 2018 MediaEval Workshop, Sophia Antipolis, France, 29/10/18. <http://ceur-ws.org/Vol-2283/>

Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection. / Cabrera-Quiros, Laura; Gedik, Ekin; Hung, Hayley.
MediaEval 2018 Multimedia Benchmark Workshop: Working Notes Proceedings of the MediaEval 2018 Workshop Sophia Antipolis, France, 29-31 October 2018. ed. / Martha Larson; Claire-Hélène Demarty; Piyush Arora. CEUR-WS.org, 2018. 3 (CEUR Workshop Proceedings; Vol. 2283).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection

AU - Cabrera-Quiros, Laura

AU - Gedik, Ekin

AU - Hung, Hayley

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper presents the algorithms that task organisers deployed for the automatic Human Behaviour Analysis (HBA) task of the MediaEval 2018. HBA task aims to investigate alternate modalities of video and body-worn acceleration for the detection of speaking status. For unimodal estimation from acceleration, a transfer learning approach, Transductive Parameter Transfer (TPT), which is shown to perform satisfactorily in a similar setting[4] is employed. For the estimation from the video modality, bags of Dense Trajectories were used in a multiple instance learning approach (MILES) [2]. Finally, late fusion is used for combining the outputs from both modalities. The multi-modal approach resulted in a mean AUC of 0.658, outperforming the performance of both single modality approaches. Copyright held by the owner/author(s).

AB - This paper presents the algorithms that task organisers deployed for the automatic Human Behaviour Analysis (HBA) task of the MediaEval 2018. HBA task aims to investigate alternate modalities of video and body-worn acceleration for the detection of speaking status. For unimodal estimation from acceleration, a transfer learning approach, Transductive Parameter Transfer (TPT), which is shown to perform satisfactorily in a similar setting[4] is employed. For the estimation from the video modality, bags of Dense Trajectories were used in a multiple instance learning approach (MILES) [2]. Finally, late fusion is used for combining the outputs from both modalities. The multi-modal approach resulted in a mean AUC of 0.658, outperforming the performance of both single modality approaches. Copyright held by the owner/author(s).

UR - https://www.scopus.com/pages/publications/85059854274

M3 - Conference contribution

AN - SCOPUS:85059854274

T3 - CEUR Workshop Proceedings

BT - MediaEval 2018 Multimedia Benchmark Workshop

A2 - Larson, Martha

A2 - Demarty, Claire-Hélène

A2 - Arora, Piyush

PB - CEUR-WS.org

T2 - 2018 MediaEval Workshop

Y2 - 29 October 2018 through 31 October 2018

ER -

Cabrera-Quiros L, Gedik E, Hung H. Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection. In Larson M, Demarty CH, Arora P, editors, MediaEval 2018 Multimedia Benchmark Workshop: Working Notes Proceedings of the MediaEval 2018 Workshop Sophia Antipolis, France, 29-31 October 2018. CEUR-WS.org. 2018. 3. (CEUR Workshop Proceedings).

Transductive parameter transfer, bags of dense trajectories and MILES for no-audio multimodal speech detection

Abstract

Publication series

Conference

Funding

Access to Document

Other files and links

Fingerprint

Cite this