Matsui Akira, Chen Emily, Wang Yunwen, Ferrara Emilio
Department of Computer Science, University of Southern California, Los Angeles, California, United States.
Information Sciences Institute, University of Southern California, Marina del Rey, California, United States.
PeerJ. 2021 Sep 15;9:e11999. doi: 10.7717/peerj.11999. eCollection 2021.
The peer-reviewing process has long been regarded as an indispensable tool in ensuring the quality of a scientific publication. While previous studies have tried to understand the process as a whole, not much effort has been devoted to investigating the determinants and impacts of the content of the peer review itself. This study leverages open data from nearly 5,000 PeerJ publications that were eventually accepted. Using sentiment analysis, Latent Dirichlet Allocation (LDA) topic modeling, mixed linear regression models, and logit regression models, we examine how the peer-reviewing process influences the acceptance timeline and contribution potential of manuscripts, and what modifications were typically made to manuscripts prior to publication. In an open review paradigm, our findings indicate that peer reviewers' choice to reveal their names in lieu of remaining anonymous may be associated with more positive sentiment in their review, implying possible social pressure from name association. We also conduct a taxonomy of the manuscript modifications during a revision, studying the words added in response to peer reviewer feedback. This study provides insights into the content of peer reviews and the subsequent modifications authors make to their manuscripts.
同行评审过程长期以来一直被视为确保科学出版物质量的不可或缺的工具。虽然先前的研究试图从整体上理解这一过程，但在调查同行评审本身内容的决定因素和影响方面投入的精力并不多。本研究利用了近5000篇最终被接受的PeerJ出版物的公开数据。通过情感分析、潜在狄利克雷分配（LDA）主题建模、混合线性回归模型和逻辑回归模型，我们研究了同行评审过程如何影响稿件的接受时间线和贡献潜力，以及在出版前稿件通常会进行哪些修改。在开放评审模式下，我们的研究结果表明，同行评审员选择公开自己的姓名而非保持匿名，可能与他们评审中更积极的情感有关，这意味着姓名关联可能带来潜在的社会压力。我们还对手稿修订过程中的修改进行了分类，研究了针对同行评审员反馈添加的词语。本研究为同行评审的内容以及作者随后对手稿所做的修改提供了见解。
