In advance of the Recognizing Preprint Peer Review meeting in December 2022, ASAPbio convened two Working Groups of different stakeholders to articulate the added value of incorporating preprint review into assessment processes, catalog potential uses of preprint review as part of assessment processes, develop working definitions for preprint review according to the needs of different stakeholders and discuss how policies and processes may be updated to incorporate preprint review. This report outlines the recommendations developed by the Working Group involving representatives of journals and preprint review projects.
Download PDF and cite as: Bertozzi, Stefano, Bloom, Theodora, Bourguet, Dennis, Brown, Katherine, Dawson, Stephanie, Edmunds, Scott, Guillemaud, Thomas, Hurst, Phil, Lacy, Michael, LaFlamme, Marcel, Lemberger, Thomas, Lumb, Elliott, MacCallum, Catriona, Pattinson, Damian, Polka, Jessica, Saderi, Daniela, Stell, Brandon, Swaminathan, Sowmya, & Puebla, Iratxe. (2023). Recommendations on Recognizing Preprint Review from the ASAPbio Journals & Preprint Review Projects Working Group. Zenodo. https://doi.org/10.5281/zenodo.7584240
The growth in use of preprints in the life sciences has opened up opportunities for innovation in the evaluation, discussion and review of research works. In the last few years, different projects and communities have started to provide comments and reviews of preprints, experimenting with a variety of models: readers may comment on preprints via the commenting features enabled by some preprint servers, or via platforms that allow commenting on the full preprint or only parts of it (e.g., PREreview, PubPeer), while some projects coordinate the review of preprints via a process similar to traditional journal peer review (e.g., Review Commons, Peer Community In, PeerRef).
Preprint feedback or preprint review?
We recognize that reactions and assessments on preprints can be very diverse in scope and format. For the purpose of these recommendations, the Working Group used the following definitions:
Preprint feedback: Any public commentary on a preprint that adds to scholarship by providing an evaluation of aspects of the study or parts of the manuscript.
Preprint review: A subset of preprint feedback generated via a process that provides transparency on the integrity of the evaluation and includes, at a minimum: 1) an assessment of the rigor and validity of the research and 2) verifiable information (direct or indirect) about the reviewer that allows their expertise and any competing interests to be checked.
Why incorporate preprint reviews into journals’ editorial processes?
Preprint feedback and reviews provide many benefits for all who engage with preprints, journals can also benefit in the following ways from incorporating preprint reviews into their editorial processes:
- Preprint review ‘democratizes’ feedback, allowing any reader to participate in peer review, which can increase diversity and provide a broader pool of potential reviewers for editors
- Greater scrutiny on the manuscript, increasing transparency and providing opportunities to identify issues not covered by the journal’s review
- Potential for quicker editorial decisions once the manuscript is submitted to the journal by using the review(s) on the preprint
- Efficiency via the reuse of the same review by different journals, reducing reviewer burden
All of these outcomes can also be beneficial to authors and reviewers themselves in addition to editors and publishers.
Incorporating preprint feedback and reviews into editorial processes
The Working Group identified four possible journal scenarios where preprint feedback and reviews may be incorporated into editorial processes: informing whether to send a manuscript for review, deciding whether a manuscript is worth highlighting, informing editorial decisions after review (i.e. complementing journal-solicited reviews), and using preprints reviews as transferred reviews within the journal’s process (i.e. replacing journal-solicited reviews). The Working Group then considered the elements needed to enable use of preprint feedback at journals: the table below outlines the elements of preprint feedback required for its use as part of the journal’s process in each scenario.
| Preprint feedback elements that may optimize use for different scenarios | Informing whether to send a paper for review | Deciding to highlight a paper (e.g. curation, commissioning articles) | Informing editorial decisions after review | Using preprint reviews as transferred reviews |
| Public review reports (with date) | ✓ | ✓ | ✓ | ✓ |
| Reviewer identity via:- editor role that verifies identity & COIs, or- self-initiated signed review + mechanism for identity verification (e.g. via ORCID) | ✓ | ✓ | ✓ | ✓ |
| Assessment of rigor & validity of the research | ✓* | ✓ | ||
| Assessment of the novelty of the work, relevance, advance in the field | ✓ | ✓ | ||
| Author initiated Editor/coordinator role Preprint version Author response Open research outputs Structured assessment | ✓ | |||
| Examples of feedback at preprint-review platforms that fulfill the requirements |  |  |  (* may vary across reviews on PREreview, the templates encourage an assessment of rigor & validity but this is not a requirement) |     |
Considering the different options available to engage with preprint reviews, there are already examples of journals that are permitting this type of activity or even engaging with it more directly.
Most journals allow preprints and have no articulated objection to authors receiving feedback or reviews on their preprint. In addition, several journals have partnerships with one or more preprint review platforms to transfer and reuse preprint reviews, examples include the Review Commons affiliate journals, the Peer Community In friendly journals, and the PeerRef partner journals.
Considerations & open questions
While developing these recommendations, the Working Group acknowledged that the approach to preprint review may vary across publishers and journals, and that additional technical tools may be needed to support adoption of preprint review by journals. We discuss some items that are likely to require consideration by journals seeking to engage with preprint review:
Who counts as a peer?
Views as to who can be designated as a peer, and thus considered qualified to review a paper will vary from one journal to another. Some journals strictly require a depth of expertise proximate to the paper’s topic as part of their reviewer selection, while others seek to include a broader range of perspectives, such as technical reviewers, patients or advocacy groups. Journals may consider whether or not to accept a specific review based on their assessment of the expertise of the person providing feedback and the types of input they typically seek as part of their review process. Such considerations are likely to influence whether journals are willing to broadly engage with preprint feedback, or only with reviews by those who meet their requirements for a peer reviewer.
Self-initiated vs platform-coordinated preprint reviews
We recognize the value of diverse contributions to commentary on preprints, and that platforms that allow individuals to contribute self-initiated feedback facilitate greater diversity in this discourse. At the same time, implementation on the journal side is more likely to take place, at least in the initial stages, in the context of preprint reviews provided by platforms that have a structured process (e.g. that include consideration of reviewer expertise, check for competing interests, or enable an author response).
Barriers to incorporate preprint reviews into journal processes
It can be difficult in the current ecosystem to track all possible comments or reviews associated with a preprint. For preprint reviews to be incorporated into journal processes, locating them must incur minimal additional burden for journal editors, reviewers and authors. Tools that facilitate automated transfer of preprint reviews into journal platforms are needed to enable broader adoption.
In addition to technical solutions, it will be important to gain a greater understanding of what elements of preprint reviews and the processes behind them can help build trust among journal editors.
Engaging with preprint reviews – a few steps for journals
We suggest a few steps that journals can take to engage with preprint review, and which can be implemented based on existing frameworks and tools:
- Develop and publicly share editorial policies (see policy template below) that articulate the journal’s support for preprint feedback and review, and how the journal uses preprint reviews.
- Allow authors to share any reviews they have received on their preprint via the journal’s submission process, and surface such reviews to the handling editor.
- Partner with a preprint-review platform, such as Review Commons, Peer Community In, PREreview or PeerRef, taking into account the frameworks the various platforms operate under and any specific requirements for peer review at the journal.
Appendix: Sample editorial policy text
{Journal name} allows authors to share their research manuscripts on preprint servers, and encourages open communications about preprints between researchers, both on preprint servers and preprint-commenting platforms. When submitting to {Journal name}, authors are welcome to share comments or reviews they have received on their preprint; this information may be provided in the cover letter accompanying the submission {(if applicable) or via the submission form on the manuscript-submission platform}. The comments and/or reviews on the preprint may be used during the editorial process for the manuscript at the editor’s discretion.
(When there is partnership with a preprint-review platform) {Journal name} has a partnership with {preprint-review platform}, enabling authors who had their preprint reviewed by {preprint-review platform} to submit the manuscript and the reviews for consideration. The {Journal name} editors will use the preprint reviews transferred from {preprint-review platform} to inform their editorial decision without restarting the review process.
Working Group Participants
- Stefano Bertozzi, Rapid Reviews: Infectious Diseases, 0000-0002-1723-7085
- Theodora Bloom, BMJ, 0000-0002-0222-4177
- Dennis Bourguet, Peer Community In, 0000-0002-2109-5323
- Katherine Brown, Company of Biologists, 0000-0001-9110-8276
- Stephanie Dawson, ScienceOpen, 0000-0002-2298-2418
- Scott Edmunds, GigaScience, 0000-0001-6444-1436
- Thomas Guillemaud, Peer Community In, 0000-0003-0451-1644
- Phil Hurst, Royal Society ,0000-0001-7971-1005
- Michael Lacy, ASCB MBoC Preprint Highlights, 0000-0003-0498-2817
- Marcel LaFlamme, PLOS, 0000-0002-7489-4233
- Thomas Lemberger, EMBO, 0000-0002-2499-4025
- Elliott Lumb, PeerRef, 0000-0001-8703-6097
- Catriona MacCallum, Hindawi, 0000-0001-9623-2225
- Damian Pattinson, eLife, 0000-0002-2606-6599
- Jessica Polka, ASAPbio, 0000-0001-6610-9293
- Daniela Saderi, PREreview, 0000-0002-6109-0367
- Brandon Stell, PubPeer, 0000-0003-1287-9211
- Sowmya Swaminathan, Springer Nature, 0000-0002-0285-4910
- Iratxe Puebla, ASAPbio, 0000-0003-1258-0746
1 Comment