Are you being rated? The effect of different rating formats on consumer experiences on online peer-to-peer platforms

We are quickly moving to a review economy. Consumers continuously rate products, services, employees, and each other on quantitative, close-ended scales (e.g., 5-point scales). Prior work on this topic has predominantly investigated how one-time exposures to accumulated product ratings affect purchase decisions (e.g., average scores on Amazon). However, especially with the advent of online peer-to-peer platforms like Uber, AirBnB and Eatwith, consumers are increasingly exposed to sequences of ratings pertaining to the same entity (i.e., product, service, or individual) on the same platform – e.g., an individual receiving sequential ratings after each Uber ride.
The dominant pattern of ratings for a service or individual on these platforms can be characterized as “mostly very good-occasionally bad”. The sparse work conducted on sequential ratings does not explain how consumers respond to such sequences, nor investigates the effects of two popular presentation formats of ratings in these sequences: a cumulative system (i.e., each rating presents an updated summary score, like an average: 5 – 5 – 5 – 5 – 4.2) and an incremental system (i.e., each rating presents the most recent score: 5 – 5 – 5 – 5 – 1). This paper enters this unchartered territory and has three contributions. First, we add to the literature on online ratings; we build on theory based on salience and numerical cognition, and empirically document situations in which a seemingly innocuous choice of rating format affects consumer judgments of typical rating patterns. Second, we contribute to the broader literature on quantitative decision making. Much of this work has focused on single or simultaneous evaluations of attributes and neglected to focus on sequentially presented, quantitative information about the same focal entity. Third, we illustrate an easy-to-change factor which can be used as a managerial tool for consumer management and prioritization.
In a series of six (mostly preregistered) studies, we find that when a positively rated entity recently received a negative score, consumers display more positive evaluations merely because a cumulative rating system is used instead of an incremental system. We show that this effect is robust across a wide set of contexts and may even lead individuals to make counter-normative judgments. That is, changing the rating system may hurt consumers’ ability to distinguish an objectively better from a weaker overall performance. We further show that the effect reverses for less recent negative ratings. In such a situation, an incremental format yields more positive evaluations than a cumulative format. Finally, we find that the strength of the effect depends on whether the cumulative rating system pertains to averages (e.g., 5 – 5 – 4) or sums (e.g., 5 – 10 – 12).
Managerially, we show that merely by altering the presentation format of a one-time low score, companies can guard consumers against overly strong responses. From a business point-of-view, the format can equally be used to either prevent negative consumer reactions (i.e., churn) or stimulate positive responses (i.e., retention). This makes the presentation format a powerful tool to control consumers’ online experiences.