Lab testing beyond usability: challenges and recommendations for assessing user experiences

Carine Lallemand, Vincent Koenig

Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

Uittreksel

In the “third wave” of human-computer interaction (HCI), the advent of the conceptual approach of UX broadens and changes the HCI landscape. Methods approved before, mainly within the conceptual approach of usability, are still widely used, and yet their adequacy for UX evaluation remains uncertain in many applications. Laboratory testing is undoubtedly the most prominent example of such a method. Hence, in this study, we investigated how the more comprehensive and emotional scope of UX can be assessed by laboratory testing.

In this paper, we report on a use case study involving 70 participants. They first took part in user/laboratory tests and then were asked to evaluate their experience with the two systems (perceived UX) by filling out an AttrakDiff scale and a UX needs fulfillment questionnaire. We conducted post-test interviews to better understand participants’ experiences. We analyzed how the participants’ perceived UX depends on quantitative (e.g., task completion time, task sequence, level of familiarity with the system) and qualitative aspects (think aloud, debriefing interviews) within the laboratory context.

Results indicate that the laboratory setting has a strong impact on the participants’ perceived UX, and support a discussion of the quality and limitations of laboratory evaluations regarding UX assessment. In this paper, we have identified concrete challenges and have provided solutions and tips useful for both practitioners and researchers who seek to account for the subjective, situated, and temporal nature of the UX in their assessments.
TaalEngels
Pagina's133-154
Aantal pagina's22
TijdschriftJournal of Usability Studies
Volume12
Nummer van het tijdschrift3
StatusGepubliceerd - mei 2017
Extern gepubliceerdJa

Trefwoorden

    Citeer dit

    @article{7a1e6d596ca64e539497382933a26ced,
    title = "Lab testing beyond usability: challenges and recommendations for assessing user experiences",
    abstract = "In the “third wave” of human-computer interaction (HCI), the advent of the conceptual approach of UX broadens and changes the HCI landscape. Methods approved before, mainly within the conceptual approach of usability, are still widely used, and yet their adequacy for UX evaluation remains uncertain in many applications. Laboratory testing is undoubtedly the most prominent example of such a method. Hence, in this study, we investigated how the more comprehensive and emotional scope of UX can be assessed by laboratory testing.In this paper, we report on a use case study involving 70 participants. They first took part in user/laboratory tests and then were asked to evaluate their experience with the two systems (perceived UX) by filling out an AttrakDiff scale and a UX needs fulfillment questionnaire. We conducted post-test interviews to better understand participants’ experiences. We analyzed how the participants’ perceived UX depends on quantitative (e.g., task completion time, task sequence, level of familiarity with the system) and qualitative aspects (think aloud, debriefing interviews) within the laboratory context.Results indicate that the laboratory setting has a strong impact on the participants’ perceived UX, and support a discussion of the quality and limitations of laboratory evaluations regarding UX assessment. In this paper, we have identified concrete challenges and have provided solutions and tips useful for both practitioners and researchers who seek to account for the subjective, situated, and temporal nature of the UX in their assessments.",
    keywords = "user experience, user testing, evaluation, laboratory evaluation, psychological needs",
    author = "Carine Lallemand and Vincent Koenig",
    year = "2017",
    month = "5",
    language = "English",
    volume = "12",
    pages = "133--154",
    journal = "Journal of Usability Studies",
    issn = "1931-3357",
    publisher = "Association for Computing Machinery, Inc",
    number = "3",

    }

    Lab testing beyond usability: challenges and recommendations for assessing user experiences. / Lallemand, Carine; Koenig, Vincent.

    In: Journal of Usability Studies, Vol. 12, Nr. 3, 05.2017, blz. 133-154.

    Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

    TY - JOUR

    T1 - Lab testing beyond usability: challenges and recommendations for assessing user experiences

    AU - Lallemand,Carine

    AU - Koenig,Vincent

    PY - 2017/5

    Y1 - 2017/5

    N2 - In the “third wave” of human-computer interaction (HCI), the advent of the conceptual approach of UX broadens and changes the HCI landscape. Methods approved before, mainly within the conceptual approach of usability, are still widely used, and yet their adequacy for UX evaluation remains uncertain in many applications. Laboratory testing is undoubtedly the most prominent example of such a method. Hence, in this study, we investigated how the more comprehensive and emotional scope of UX can be assessed by laboratory testing.In this paper, we report on a use case study involving 70 participants. They first took part in user/laboratory tests and then were asked to evaluate their experience with the two systems (perceived UX) by filling out an AttrakDiff scale and a UX needs fulfillment questionnaire. We conducted post-test interviews to better understand participants’ experiences. We analyzed how the participants’ perceived UX depends on quantitative (e.g., task completion time, task sequence, level of familiarity with the system) and qualitative aspects (think aloud, debriefing interviews) within the laboratory context.Results indicate that the laboratory setting has a strong impact on the participants’ perceived UX, and support a discussion of the quality and limitations of laboratory evaluations regarding UX assessment. In this paper, we have identified concrete challenges and have provided solutions and tips useful for both practitioners and researchers who seek to account for the subjective, situated, and temporal nature of the UX in their assessments.

    AB - In the “third wave” of human-computer interaction (HCI), the advent of the conceptual approach of UX broadens and changes the HCI landscape. Methods approved before, mainly within the conceptual approach of usability, are still widely used, and yet their adequacy for UX evaluation remains uncertain in many applications. Laboratory testing is undoubtedly the most prominent example of such a method. Hence, in this study, we investigated how the more comprehensive and emotional scope of UX can be assessed by laboratory testing.In this paper, we report on a use case study involving 70 participants. They first took part in user/laboratory tests and then were asked to evaluate their experience with the two systems (perceived UX) by filling out an AttrakDiff scale and a UX needs fulfillment questionnaire. We conducted post-test interviews to better understand participants’ experiences. We analyzed how the participants’ perceived UX depends on quantitative (e.g., task completion time, task sequence, level of familiarity with the system) and qualitative aspects (think aloud, debriefing interviews) within the laboratory context.Results indicate that the laboratory setting has a strong impact on the participants’ perceived UX, and support a discussion of the quality and limitations of laboratory evaluations regarding UX assessment. In this paper, we have identified concrete challenges and have provided solutions and tips useful for both practitioners and researchers who seek to account for the subjective, situated, and temporal nature of the UX in their assessments.

    KW - user experience

    KW - user testing

    KW - evaluation

    KW - laboratory evaluation

    KW - psychological needs

    M3 - Article

    VL - 12

    SP - 133

    EP - 154

    JO - Journal of Usability Studies

    T2 - Journal of Usability Studies

    JF - Journal of Usability Studies

    SN - 1931-3357

    IS - 3

    ER -