Save the last dance for me: Unwanted serial position effects in jury evaluations

Whenever competing options are considered in sequence, their evaluations may be affected by order of appearance. Such serial position effects would threaten the fairness of competitions using jury evaluations. Randomization cannot reduce potential order effects, but it does give candidates an equal chance of being assigned to preferred serial positions. Whether, or what, serial position effects emerge may depend on the cognitive demands of the judgment task. In end-of-sequence procedures, final scores are not given until all candidates have performed, possibly burdening judges’ memory. If judges’ evaluations are based on how well they remember performances, serial position effects may resemble those found with free recall. Candidates may also be evaluated step-by-step, immediately after each performance. This procedure should not burden memory, though it may produce different serial position effects. Yet, this paper reports similar serial position effects with end-of-sequence and step-by-step procedures used for the Eurovision Song Contest: Ratings increased with serial position. The linear order effect was replicated in the step-by-step judgments of World and European Figure Skating Contests. It is proposed that, independent of the evaluation procedure, judges’ initial impressions of sequentially appearing candidates may be formed step-by-step, yielding serial position effects.