Powerful and consistent analysis of likert-type ratingscales

Likert-type scales are used extensively during usability evaluations, and more generally evaluations of interactive experiences, to obtain quantified data regarding attitudes, behaviors, and judgments of participants. Very often this data is analyzed using parametric statistics like the Student t-test or ANOVAs. These methods are chosen to ensure higher statistical power of the test (which is necessary in this field of research and practice where sample sizes are often small), or because of the lack of software to handle multi-factorial designs nonparametrically. With this paper we present to the HCI audience new developments from the field of medical statistics that enable analyzing multiple factor designs nonparametrically. We demonstrate the necessity of this approach by showing the errors in the parametric treatment of nonparametric data in experiments of the size typically reported in HCI research. We also provide a practical resource for researchers and practitioners who wish to use these new methods.