Productivity and patterns of activity in bug bounty programs: analysis of hackerone and Google vulnerability research

In this work, we considered two well-known bug bounty programs - HackerOne and Google Vulnerability Research - with the goal of investigating patterns of activity and comparing productivity of security researchers. HackerOne and Google’s programs differ in many ways. HackerOne is one of the largest and most successful bug bounty programs, with heterogeneous membership of security researchers and software producers. Google Vulnerability Research, instead, is a closed program for selected Google employees working on a more homogeneous range of software. For the analysis, we introduced three productivity metrics, which let us study the performance of researchers under different perspectives and possible patterns of activity. A contribution of this work is to shed new light on the yet not well understood environment represented by bug bounties and software vulnerability discovery initiatives. The low-hanging fruits approach adopted by unexperienced researchers in open bug bounties has been often discussed, but less is known about the approach adopted by more experienced participants. Another result is to have shown that a generic comparison between different bug bounty programs may lead to wrong conclusions. Bug bounty programs could exhibits large variations in researcher profiles and software characteristics, which make them not comparable without a careful examination of homogeneous subsets of participants and incentive mechanisms.