Autonomy Is An Acquired Taste: Exploring Developer Preferences for GitHub Bots

Software bots fulfill an important role in collective software development, and their adoption by developers promises increased productivity. Past research has identified that bots that communicate too often can irritate developers, which affects the utility of the bot. However, it is not clear what other properties of human-bot collaboration affect developers’ preferences, or what impact these properties might have. The main idea of this paper is to explore characteristics affecting developer preferences for interactions between humans and bots, in the context of GitHub pull requests. We carried out an exploratory sequential study with interviews and a subsequent vignette-based survey. We find developers generally prefer bots which are personable but show little autonomy, however more experienced developers tend to prefer more autonomous bots. Based on our empirical evidence we recommend bot developers increase configuration options for bots such that individual developers and projects can configure bots to best align with their own preferences and project culture.