Peiling Wang, Professor, University of Tennessee-Knoxville & Dietmar Wolfram, Professor, University of Wisconsin-Milwaukee
In a recent open-access (OA) peer-reviewed article1, we report on the rapid growth of open peer review (OPR) journals from 38 in 2001 to 617 in 2019 with a significant increase since 2017 (the dataset2 is available at Zenodo). One of the major contributing publishers is MDPI which lists 204 OPR journals, with 128 and 63 OPR titles added in 2018 and 2019, respectively. Another significant contributor is the publisher SCIENCE DOMAIN International, of the 111 OPR journals it publishes, 59 began publication from 2018. In addition to these OA publishers, major longstanding publishers with OPR titles should be recognized as well, of the 70 OPR journals published by BioMed Central, 36 titles were among the first OPR journals, operating in that model since their launch in 2001. Wiley pioneered one OPR journal in 2016, added 39 OPR journals in 2018, and will have a total of 50 OPR journals in 2020. Publishers such as the BMJ, Elsevier, and Nature all published at least one OPR journal by 2019; Nature will add seven more journals in addition to Nature Communications (which implemented that review modality in 2016) to its OPR journal list in 2020. These trends are encouraging and signal an increasing interest in further transparency in the peer review process.
Since 2001, different OPR models have been adopted and implemented, representing various levels of transparency and openness: from publishing only reviewer identities (e.g., Frontiers in journals) to real-time process tracking with review reports and reviewer identities appearing alongside the paper as it undergoes review (e.g., F1000Research). Since its first publications in 2013, PeerJ has published optional review histories (author’s choice) and reviewer identities (reviewer’s choice).
Figure 3 in Wolfram, D., Wang, P., Hembree, A. & Park, H. (2020) Open peer review: promoting transparency in open science. Scientometrics.
Over the past five years, we have been closely following developments and trends in different aspects of OPR. At the panel “The last frontier in open science: Will open peer review transform scientific and scholarly publishing?” at the Association for Information Science and Technology 2016 Annual Meeting (ASIS&T2016): we asked the attendees: As a reviewer, would you be willing to make your identity public along with the review content alongside a published article? As an author, would you make public the peer review comments you received? As a reader of a published article, would you find value in reading peer review comments that accompany an article? These are the questions several studies, including the 2018 ASAPbio surveys3, have been trying to gauge.
Our 2017 web-mining study of PeerJpublications4 found that approximately 74% of articles had associated full review histories. Of the articles with published reviews, 18% listed identities for all the reviewers, and 53% had at least one signed review. In total, 43% of all posted reviews were signed.
We are continuing to investigate different aspects of OPR using both quantitative and qualitative methods. Currently, we are using text mining and sentiment analysis strategies to analyze reviewer comments collected from OPR journals with the goal of evaluating the quality of peer reviews and the language used by the reviewers. Further, we plan to investigate how researchers are using published peer reviews.
As more journals implement OPR, the increasing number of available published reviews will open up opportunities for further studies to address author, reviewer and reader practices as they engage with peer review.
Acknowledgements: This project was partially supported by a University of Wisconsin-Milwaukee Research Growth Initiative Grant, 2018-2019. The open-access APC was in part supported by a 2020 Dean’s Summer Research grant to Peiling Wang.
Wolfram, D., Wang, P., Hembree, A. & Park, H. (2020) Open peer review: promoting transparency in open science. Scientometrics. https://doi.org/10.1007/s11192-020-03488-4
Peiling Wang, Professor, University of Tennessee-Knoxville & Dietmar Wolfram, Professor, University of Wisconsin-Milwaukee
In a recent open-access (OA) peer-reviewed article1, we report on the rapid growth of open peer review (OPR) journals from 38 in 2001 to 617 in 2019 with a significant increase since 2017 (the dataset2 is available at Zenodo). One of the major contributing publishers is MDPI which lists 204 OPR journals, with 128 and 63 OPR titles added in 2018 and 2019, respectively. Another significant contributor is the publisher SCIENCE DOMAIN International, of the 111 OPR journals it publishes, 59 began publication from 2018. In addition to these OA publishers, major longstanding publishers with OPR titles should be recognized as well, of the 70 OPR journals published by BioMed Central, 36 titles were among the first OPR journals, operating in that model since their launch in 2001. Wiley pioneered one OPR journal in 2016, added 39 OPR journals in 2018, and will have a total of 50 OPR journals in 2020. Publishers such as the BMJ, Elsevier, and Nature all published at least one OPR journal by 2019; Nature will add seven more journals in addition to Nature Communications (which implemented that review modality in 2016) to its OPR journal list in 2020. These trends are encouraging and signal an increasing interest in further transparency in the peer review process.
Since 2001, different OPR models have been adopted and implemented, representing various levels of transparency and openness: from publishing only reviewer identities (e.g., Frontiers in journals) to real-time process tracking with review reports and reviewer identities appearing alongside the paper as it undergoes review (e.g., F1000Research). Since its first publications in 2013, PeerJ has published optional review histories (author’s choice) and reviewer identities (reviewer’s choice).
Over the past five years, we have been closely following developments and trends in different aspects of OPR. At the panel “The last frontier in open science: Will open peer review transform scientific and scholarly publishing?” at the Association for Information Science and Technology 2016 Annual Meeting (ASIS&T2016): we asked the attendees: As a reviewer, would you be willing to make your identity public along with the review content alongside a published article? As an author, would you make public the peer review comments you received? As a reader of a published article, would you find value in reading peer review comments that accompany an article? These are the questions several studies, including the 2018 ASAPbio surveys3, have been trying to gauge.
Our 2017 web-mining study of PeerJ publications4 found that approximately 74% of articles had associated full review histories. Of the articles with published reviews, 18% listed identities for all the reviewers, and 53% had at least one signed review. In total, 43% of all posted reviews were signed.
We are continuing to investigate different aspects of OPR using both quantitative and qualitative methods. Currently, we are using text mining and sentiment analysis strategies to analyze reviewer comments collected from OPR journals with the goal of evaluating the quality of peer reviews and the language used by the reviewers. Further, we plan to investigate how researchers are using published peer reviews.
As more journals implement OPR, the increasing number of available published reviews will open up opportunities for further studies to address author, reviewer and reader practices as they engage with peer review.
Acknowledgements: This project was partially supported by a University of Wisconsin-Milwaukee Research Growth Initiative Grant, 2018-2019. The open-access APC was in part supported by a 2020 Dean’s Summer Research grant to Peiling Wang.