Samenvatting
Software engineering researchers have been using general purpose online tools for crowd-sourcing for quite some time. Those tools can be useful to recruit participants for research studies as they are paid for their time. However, those tools should be used carefully. In this paper, we have described the issues we faced when recruiting participants on Prolific.
We used Prolific to recruit open-source developers with experience in submitting and reviewing pull requests (PRs). However, we did not succeed in obtaining valid participants for either the interview or the survey, which led us to change the approach of our study and not use Prolific anymore. For example, a major issue we faced on Prolific was that even using the specific filter related to participants with software development knowledge, the screening study revealed the majority of them did not have any development knowledge. Based on this experience, we formulate recommendations for future research when using such crowd-sourcing tools.
We used Prolific to recruit open-source developers with experience in submitting and reviewing pull requests (PRs). However, we did not succeed in obtaining valid participants for either the interview or the survey, which led us to change the approach of our study and not use Prolific anymore. For example, a major issue we faced on Prolific was that even using the specific filter related to participants with software development knowledge, the screening study revealed the majority of them did not have any development knowledge. Based on this experience, we formulate recommendations for future research when using such crowd-sourcing tools.
Originele taal-2 | Engels |
---|---|
Status | Geaccepteerd/In druk - 19 feb. 2022 |
Evenement | International Workshop on Recruiting Participants for Empirical Software Engineering - Virtual Duur: 17 mei 2022 → 17 mei 2022 https://ropes-workshops.github.io/ropes22/ |
Congres
Congres | International Workshop on Recruiting Participants for Empirical Software Engineering |
---|---|
Verkorte titel | RoPES |
Periode | 17/05/22 → 17/05/22 |
Internet adres |