Self-consistent peer ranking for assessing student work : dealing with large populations

C.W.A.M. Overveld, van, T. Verhoeff

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)


Assessing large populations of students puts a serious burden on teaching staff capacity. For open-format assignments, automation of the reviewing process can offer only limited support. Peer ranking is a partial solution to the problem, with the added benefit that students' critical reading skills are developed. We see two remaining problems, however: (1) for students, it is a major challenge to assign marks on an absolute scale, and (2) students' competence in reviewing may vary significantly - so not all peer reviews should have a similar weight in the process. To remedy these shortcomings, we suggest an approach to peer ranking, inspired by Jon Kleinberg's FHTS-algorithm, where both the students' assignment results and the quality of their double anonymous peer reviews are algorithmically ranked. Based on preliminary model calculations, we estimate that this strategy may reduce the required effort for reviewing open-format assignments approximately by a factor of ten. A first large-scale pilot with this method will take place in undergraduate courses at Eindhoven University of Technology, spring 2013. Since this involves about 900 students, automated support is a must. We describe the peer reviewing facilities that were introduced in our web-based education support system named Peach.
Original languageEnglish
Title of host publication5th International Conference on Computer Supported Education (CSEDU 2013), May 6-8, 2013, Aachen, Germany
EditorsO. Foley, M.T. Restivo, J. Onohuome Uhomoibhi, M. Helfert
Place of PublicationAachen, Germany
PublisherSciTePress Digital Library
ISBN (Print)978-989856553-2
Publication statusPublished - 2013
Event5th International Conference on Computer Supported Education (CSEDU 2013) - Aachen, Germany
Duration: 6 May 20138 May 2013
Conference number: 5


Conference5th International Conference on Computer Supported Education (CSEDU 2013)
Abbreviated titleCSEDU 2013
Internet address


Dive into the research topics of 'Self-consistent peer ranking for assessing student work : dealing with large populations'. Together they form a unique fingerprint.

Cite this