Modelling and detecting changes in user satisfaction

J. Kiseleva, E. Crestan, R. Brigo, R. Dittel

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    9 Citations (Scopus)
    1 Downloads (Pure)

    Abstract

    Informational needs behind queries, that people issue to search engines, are inherently sensitive to external factors such as breaking news, new models of devices, or seasonal changes as 'black Friday'. Mostly these changes happen suddenly and it is natural to suppose that they may cause a shift in user satisfaction with presented old search results and push users to reformulate their queries. For instance, if users issued the query 'CIKM conference' in 2013 they were satisfied with results referring to the page cikm2013.org and this page gets a majority of clicks. However, the confernce site has been changed and the same query issued in 2014 should be linked to the different page cikm2014.fudan.edu.cn. If the link to the fresh page is not among the retrieved results then users will reformulate the query to find desired information. In this paper, we examine how to detect changes in user satisfaction if some events affect user information goals but search results remained the same. We formulate a problem using concept drift detection techniques. The proposed method works in an unsupervised manner, we do not rely on any labelling. We report results of a large scale evaluation over real user interactions, that are collected by a commercial search engine within six months. The final datasets consist of more than sixty millions log entries. The results of our experiments demonstrate that by using our method we can accurately detect changes in user behavior. The detected drifts can be used to enhance query auto-completion, user satisfaction metrics, and recency ranking.
    Original languageEnglish
    Title of host publicationProceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM'14, Shanghai, China, November 3-7, 2014)
    Place of PublicationNew York NY
    PublisherAssociation for Computing Machinery, Inc
    Pages1449-1458
    ISBN (Print)978-1-4503-2598-1
    DOIs
    Publication statusPublished - 2014
    Eventconference; 23rd ACM International Conference on Conference on Information and Knowledge Management; 2014-11-03; 2014-11-07 -
    Duration: 3 Nov 20147 Nov 2014

    Conference

    Conferenceconference; 23rd ACM International Conference on Conference on Information and Knowledge Management; 2014-11-03; 2014-11-07
    Period3/11/147/11/14
    Other23rd ACM International Conference on Conference on Information and Knowledge Management

    Fingerprint

    Dive into the research topics of 'Modelling and detecting changes in user satisfaction'. Together they form a unique fingerprint.

    Cite this