Erlenhov Linda, de Oliveira Neto Francisco Gomes, Leitner Philipp
Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden.
Department of Computer Science and Engineering, University of Gothenburg, Gothenburg, Sweden.
PeerJ Comput Sci. 2022 Mar 3;8:e849. doi: 10.7717/peerj-cs.849. eCollection 2022.
Bots have become active contributors in maintaining open-source repositories. However, the definitions of bot activity in open-source software vary from a more lenient stance encompassing every non-human contributions frameworks that cover contributions from tools that have autonomy or human-like traits (., Devbots). Understanding which of those definitions are being used is essential to enable (i) reliable sampling of bots and (ii) fair comparison of their practical impact in, ., developers' productivity. This paper reports on an empirical study composed of both quantitative and qualitative analysis of bot activity. By analysing those two bot definitions in an existing dataset of bot commits, we see that only 10 out of 54 listed tools (mainly dependency management) comply with the characteristics of Devbots. Moreover, five of those Devbots have similar patterns of contributions over 93 projects, such as similar proportions of merged pull-requests and days until issues are closed. Our analysis also reveals that most projects (77%) experiment with more than one bot before deciding to adopt or switch between bots. In fact, a thematic analysis of developers' comments in those projects reveal factors driving the discussions about Devbot adoption or removal, such as the impact of the generated noise and the needed adaptation in development practices within the project.
机器人已成为维护开源存储库的积极贡献者。然而，开源软件中机器人活动的定义各不相同，从较为宽松的立场（涵盖每一项非人类贡献）到包含来自具有自主性或类人特征的工具（如开发机器人）贡献的框架。了解使用了哪些定义对于实现以下两点至关重要：（i）对机器人进行可靠抽样，以及（ii）公平比较它们对开发者生产力等方面的实际影响。本文报告了一项对机器人活动进行定量和定性分析的实证研究。通过在现有的机器人提交数据集中分析这两种机器人定义，我们发现列出的54个工具中只有10个（主要是依赖项管理工具）符合开发机器人的特征。此外，其中5个开发机器人在93个项目中的贡献模式相似，例如合并拉取请求的比例和问题关闭前的天数相似。我们的分析还表明，大多数项目（77%）在决定采用或在机器人之间切换之前会试用不止一个机器人。事实上，对这些项目中开发者评论的主题分析揭示了推动关于采用或移除开发机器人讨论的因素，例如产生的噪音影响以及项目内开发实践所需的调整。
