As recommender systems are increasingly deployed in the real world, they are not merely tested offline for precision and coverage, but also "online" with test users to ensure good user experience. The user evaluation of recommenders is however complex and resource-consuming. We introduce a pragmatic procedure to evaluate recommender systems for experience products with test users, within industry constraints on time and budget. Researchers and practitioners can employ our approach to gain a comprehensive understanding of the user experience with their systems.
|Title of host publication||RecSys '11 Proceedings of the fifth ACM International Conference on Recommender Systems, 23-27 October 2011, Chicago, Il., USA|
|Editors||B. Mobasher, R. Burke|
|Place of Publication||New York|
|Publisher||Association for Computing Machinery, Inc|
|Publication status||Published - 2011|