A Systematic Literature Review of Empirical Methods and Risk Representation in Usable Privacy and Security Research

Verena DIstler; Matthias Fassl; Hana Habib; Katharina Krombholz; Gabriele Lenzini; Carine Lallemand; Lorrie Faith Cranor; Vincent Koenig

doi:10.1145/3469845

A Systematic Literature Review of Empirical Methods and Risk Representation in Usable Privacy and Security Research

Verena DIstler
, Matthias Fassl
, Hana Habib
, Katharina Krombholz
, Gabriele Lenzini
, Carine Lallemand
, Lorrie Faith Cranor
, Vincent Koenig

EAISI Health

Research output: Contribution to journal › Review article › peer-review

63 Citations (Scopus)

259 Downloads (Pure)

Abstract

Usable privacy and security researchers have developed a variety of approaches to represent risk to research participants. To understand how these approaches are used and when each might be most appropriate, we conducted a systematic literature review of methods used in security and privacy studies with human participants. From a sample of 633 papers published at five top conferences between 2014 and 2018 that included keywords related to both security/privacy and usability, we systematically selected and analyzed 284 full-length papers that included human subjects studies. Our analysis focused on study methods; risk representation; the use of prototypes, scenarios, and educational intervention; the use of deception to simulate risk; and types of participants. We discuss benefits and shortcomings of the methods, and identify key methodological, ethical, and research challenges when representing and assessing security and privacy risk. We also provide guidelines for the reporting of user studies in security and privacy.

Original language	English
Article number	43
Number of pages	50
Journal	ACM Transactions on Computer-Human Interaction
Volume	28
Issue number	6
DOIs	https://doi.org/10.1145/3469845
Publication status	Published - Dec 2021

Bibliographical note

Funding Information:
This work is supported by the Fonds National de la Recherche (PRIDE15/10621687) and the Carnegie Corporation of New York. Authors’ addresses: V. Distler, G. Lenzini, and V. Koenig, University of Luxembourg; M Fassl and K. Krombholz, University of Luxembourg Maison des Sciences Humaines 11, Porte des Sciences L-4366 Esch-sur-Alzette Luxembourg; H. Habib and L. F. Cranor, Carnegie Mellon University, 4720 Forbes Avenue, Pittsburgh, PA 15213, United States; emails: [email protected], [email protected]; C. Lallemand, Eindhoven University of Technology & University of Luxembourg.

Funding

This work is supported by the Fonds National de la Recherche (PRIDE15/10621687) and the Carnegie Corporation of New York. Authors’ addresses: V. Distler, G. Lenzini, and V. Koenig, University of Luxembourg; M Fassl and K. Krombholz, University of Luxembourg Maison des Sciences Humaines 11, Porte des Sciences L-4366 Esch-sur-Alzette Luxembourg; H. Habib and L. F. Cranor, Carnegie Mellon University, 4720 Forbes Avenue, Pittsburgh, PA 15213, United States; emails: [email protected], [email protected]; C. Lallemand, Eindhoven University of Technology & University of Luxembourg.

Keywords

Human-Computer Interaction (HCI)
Usable privacy and security
user experience (UX) research

Access to Document

10.1145/3469845

pdfFinal published version, 978 KBLicence: CC BY-NC-ND

Cite this

@article{4e34bb71e0024cf39b1334712db5c59c,

title = "A Systematic Literature Review of Empirical Methods and Risk Representation in Usable Privacy and Security Research",

abstract = "Usable privacy and security researchers have developed a variety of approaches to represent risk to research participants. To understand how these approaches are used and when each might be most appropriate, we conducted a systematic literature review of methods used in security and privacy studies with human participants. From a sample of 633 papers published at five top conferences between 2014 and 2018 that included keywords related to both security/privacy and usability, we systematically selected and analyzed 284 full-length papers that included human subjects studies. Our analysis focused on study methods; risk representation; the use of prototypes, scenarios, and educational intervention; the use of deception to simulate risk; and types of participants. We discuss benefits and shortcomings of the methods, and identify key methodological, ethical, and research challenges when representing and assessing security and privacy risk. We also provide guidelines for the reporting of user studies in security and privacy. ",

keywords = "Human-Computer Interaction (HCI), Usable privacy and security, user experience (UX) research",

author = "Verena DIstler and Matthias Fassl and Hana Habib and Katharina Krombholz and Gabriele Lenzini and Carine Lallemand and Cranor, \{Lorrie Faith\} and Vincent Koenig",

note = "Funding Information: This work is supported by the Fonds National de la Recherche (PRIDE15/10621687) and the Carnegie Corporation of New York. Authors{\textquoteright} addresses: V. Distler, G. Lenzini, and V. Koenig, University of Luxembourg; M Fassl and K. Krombholz, University of Luxembourg Maison des Sciences Humaines 11, Porte des Sciences L-4366 Esch-sur-Alzette Luxembourg; H. Habib and L. F. Cranor, Carnegie Mellon University, 4720 Forbes Avenue, Pittsburgh, PA 15213, United States; emails: [email protected], [email protected]; C. Lallemand, Eindhoven University of Technology \& University of Luxembourg. ",

year = "2021",

month = dec,

doi = "10.1145/3469845",

language = "English",

volume = "28",

journal = "ACM Transactions on Computer-Human Interaction",

issn = "1073-0516",

publisher = "Association for Computing Machinery, Inc.",

number = "6",

}

TY - JOUR

T1 - A Systematic Literature Review of Empirical Methods and Risk Representation in Usable Privacy and Security Research

AU - DIstler, Verena

AU - Fassl, Matthias

AU - Habib, Hana

AU - Krombholz, Katharina

AU - Lenzini, Gabriele

AU - Lallemand, Carine

AU - Cranor, Lorrie Faith

AU - Koenig, Vincent

N1 - Funding Information: This work is supported by the Fonds National de la Recherche (PRIDE15/10621687) and the Carnegie Corporation of New York. Authors’ addresses: V. Distler, G. Lenzini, and V. Koenig, University of Luxembourg; M Fassl and K. Krombholz, University of Luxembourg Maison des Sciences Humaines 11, Porte des Sciences L-4366 Esch-sur-Alzette Luxembourg; H. Habib and L. F. Cranor, Carnegie Mellon University, 4720 Forbes Avenue, Pittsburgh, PA 15213, United States; emails: [email protected], [email protected]; C. Lallemand, Eindhoven University of Technology & University of Luxembourg.

PY - 2021/12

Y1 - 2021/12

N2 - Usable privacy and security researchers have developed a variety of approaches to represent risk to research participants. To understand how these approaches are used and when each might be most appropriate, we conducted a systematic literature review of methods used in security and privacy studies with human participants. From a sample of 633 papers published at five top conferences between 2014 and 2018 that included keywords related to both security/privacy and usability, we systematically selected and analyzed 284 full-length papers that included human subjects studies. Our analysis focused on study methods; risk representation; the use of prototypes, scenarios, and educational intervention; the use of deception to simulate risk; and types of participants. We discuss benefits and shortcomings of the methods, and identify key methodological, ethical, and research challenges when representing and assessing security and privacy risk. We also provide guidelines for the reporting of user studies in security and privacy.

AB - Usable privacy and security researchers have developed a variety of approaches to represent risk to research participants. To understand how these approaches are used and when each might be most appropriate, we conducted a systematic literature review of methods used in security and privacy studies with human participants. From a sample of 633 papers published at five top conferences between 2014 and 2018 that included keywords related to both security/privacy and usability, we systematically selected and analyzed 284 full-length papers that included human subjects studies. Our analysis focused on study methods; risk representation; the use of prototypes, scenarios, and educational intervention; the use of deception to simulate risk; and types of participants. We discuss benefits and shortcomings of the methods, and identify key methodological, ethical, and research challenges when representing and assessing security and privacy risk. We also provide guidelines for the reporting of user studies in security and privacy.

KW - Human-Computer Interaction (HCI)

KW - Usable privacy and security

KW - user experience (UX) research

UR - https://www.scopus.com/pages/publications/85122435457

U2 - 10.1145/3469845

DO - 10.1145/3469845

M3 - Review article

AN - SCOPUS:85122435457

SN - 1073-0516

VL - 28

JO - ACM Transactions on Computer-Human Interaction

JF - ACM Transactions on Computer-Human Interaction

IS - 6

M1 - 43

ER -

A Systematic Literature Review of Empirical Methods and Risk Representation in Usable Privacy and Security Research

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this