What to Expect from Code Review Bots on GitHub? A Survey with OSS Maintainers

Mairieli Wessel, Alexander Serebrenik, Igor Wiese, Igor Steinmacher, Marco A. Gerosa

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

4 Citations (Scopus)


Software bots are used by Open Source Software (OSS) projects to streamline the code review process. Interfacing between developers and automated services, code review bots report continuous integration failures, code quality checks, and code coverage. However, the impact of such bots on maintenance tasks is still neglected. In this paper, we study how project maintainers experience code review bots. We surveyed 127 maintainers and asked about their expectations and perception of changes incurred by code review bots. Our findings reveal that the most frequent expectations include enhancing the feedback bots provide to developers, reducing the maintenance burden for developers, and enforcing code coverage. While maintainers report that bots satisfied their expectations, they also perceived unexpected effects, such as communication noise and newcomers' dropout. Based on these results, we provide a series of implications for bot developers, as well as insights for future research.

Original languageEnglish
Title of host publicationProceedings - 34th Brazilian Symposium on Software Engineering, SBES 2020
PublisherAssociation for Computing Machinery, Inc
Number of pages6
ISBN (Electronic)9781450387538
Publication statusPublished - 21 Oct 2020
Event34th Brazilian Symposium on Software Engineering, SBES 2020 - Virtual, Online, Brazil
Duration: 21 Oct 202023 Oct 2020

Publication series

NameACM International Conference Proceeding Series


Conference34th Brazilian Symposium on Software Engineering, SBES 2020
CityVirtual, Online

Bibliographical note

Funding Information:
We thank all the participants of this study, who volunteered to support our research. This work was partially supported by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001, CNPq (grant 141222/2018-2), and National Science Foundation (grants 1815503 and 1900903).

Publisher Copyright:
© 2020 ACM.


  • code review
  • open source software
  • pull-based model
  • software bots


Dive into the research topics of 'What to Expect from Code Review Bots on GitHub? A Survey with OSS Maintainers'. Together they form a unique fingerprint.

Cite this