On December 9, 2019, ASAPbio, EMBO Press, and 17 affiliate journals launched Review Commons, a platform providing independent peer review prior to journal submission. At two subsequent community meetings (one in-person at the ASCB|EMBO conference and the other online the following week) the Review Commons team shared their motivations, planned workflow, and future vision. We also heard questions, concerns, and ideas about the project, which are summarized here as a composite of the two events.
Why Review Commons?
The speed of scientific communication has not kept pace with tremendous advances in web-based technology. For example, graduates students are taking on average one year longer to publish their first author work than they were 30 years ago, and the number of figure panels per journal article has dramatically increased over the same period. One reason for these rising expectations may be that the existing journal publication system performs many different functions for academic research, including dissemination, evaluation, and curation. Despite more widespread adoption of preprints to quickly disseminate research findings, evaluation of scientists is tied to peer review via the perceived prestige of the journal they publish in. Consequently, there’s pressure to combine many results into a single “big story” in an effort to publish in a high impact journal, and some papers are serially resubmitted to many journals in an effort to find the most prestigious possible home. This inefficient process depletes valuable peer review time and resources, leading to an estimated 15 million hours of wasted reviewer time per year.
Enter Review Commons, “a small village,” in the words of Ron Vale (founder of ASAPbio), in which collaborators are working to do something good for science. Ron explained that the goal is to inspire a cultural change in which reviews are written in a spirit of partnership with the author for the improvement of science. In this model, reviewers focus on the science rather than journal fit. Moreover, Review Commons aims to make the publishing process more efficient by facilitating better match-making between scientists and journals. ASAPbio and EMBO are approaching Review Commons as an experiment in peer review.
Initial reactions to the platform
Maria Leptin (Director of EMBO) reminded the audience that the success of the project depends on engagement and adoption by the research community. Fortunately, the announcement of Review Commons was met with huge enthusiasm: during formation of the scientific advisory board, out of 109 invitations sent, 80 were accepted in less than a week, with only 2 declines. Maria acknowledged that many from the research community are wondering how Review Commons will work if the referees don’t know which journal the manuscript is destined for. This reveals that our concept of peer review is deeply entangled with journal brands; Review Commons can help to differentiate between the scientific strength of a manuscript and the editorial assessment of suitability for publication in a specific journal. However, Review Commons is a limited-time trial, and by the end of 2020, the project will have to develop a sustainable and scalable business model.
Journal-agnostic, high-quality peer review
Review Commons aims to decouple the many aspects of publishing (dissemination, evaluation, and curation) by reversing the canonical sequence of journal selection and peer review. The final output will be a peer reviewed preprint that can be submitted, with the platform’s facilitation, to one of 17 affiliate journals. The scope is currently in the life sciences, where EMBO Press is confident in obtaining high-quality peer review. Thomas Lemberger (Deputy Head of Scientific Publications and Review Commons project lead at EMBO) emphasized that reviewers will be asked to evaluate the submitted work for what it is, not what it should or can be to fit the editorial standards of a specific journal. To learn more about the work flow, please watch Thomas’ detailed description in the recording.
EMBO Press and ASAPbio will evaluate Review Commons periodically and use this information to adapt the workflows and policies.
Journals benefit from peer review focused on the science
Why engage as an affiliate journal? Bernd Pulverer (Head of Scientific Publications, EMBO) explained that Review Commons will help affiliate journals adopt a more transparent publishing process in which reviews are posted on preprints (see the first Refereed Preprint available on February 3, 2020). They will also benefit from the sequential manuscript transfer network from the platform to partner journals: since the first set of reviews is coordinated centrally by Review Commons, journals do not have to re-assign new referees for re-review of transferred manuscripts. This dramatically increases the overall efficacy of the peer review process. With Review Commons, access to review reports before submitting to a journal can help authors choose an appropriate venue, which further streamlines the editorial process. Through these measures, Review Commons can help conserve editorial resources, potentially reducing the number of reviews per paper from 9-12 by the time of publication to 3-4. Because the platform may attract a new demographic of authors interested in transparent and innovative publishing, there may be a net increase in submissions to affiliate journals. The process may be challenged by: 1) too few or too many submissions, 2) poor quality submissions leading to difficulty in securing referees, 3) referees that do not comply with requests to be journal agnostic, or 4) an uneven distribution of Review Commons transfers to certain partner journals. EMBO and ASAPbio will monitor all of these possibilities.
After the presentations in both events, audience members raised questions and made comments during Q&A periods. Some of those questions and corresponding answers from EMBO & ASAPbio are summarized below.
The review process
How does the initial triage by the Review Commons managing editor work?
The initial triage decision is made by the managing editor in consultation with the advisory board. To maintain a high-quality review process and conserve the finite resource of reviewer time, preselection at the initial triage stage is kept quick and stringent, sending only studies that represent significant advances to review.
How long will the peer review process take? Could this process be faster than the journal-organized peer review?
A round of review at Review Commons may not be faster than review at affiliate journals (approximately 28-30 days). If anything, it might be sometimes more challenging to find reviewers because the platform is new. The biggest time saver is not the duration of a round of peer review, but rather the reduction of serial submission and repetition of that process at other journals.
Why is bioRxiv the only preprint server partner?
This decision was purely practical; bioRxiv is the dominant preprint server in the life sciences, and it supports the technology that makes it possible to post the reviews. Other preprint servers may be incorporated in the future.
Could the Review Commons Refereed Preprint be a final endpoint for a paper?
Yes, there is no obligation to submit to an affiliate journal. These decisions will be tracked as a part of the analysis of the platform. Authors can also exit the Review Commons system and submit to another journal with the peer review reports they received.
Transparency and data sharing
Will cross-commenting be anonymous? Can reviewers disclose their identities to one another?
Believing that the most important component of reviews is the scientific content, Review Commons will conduct cross-commenting anonymously, unless reviewers decide to sign their reviews.
What level of transparency will there be about time to transfer, time in review, cost savings, etc?
The time of review will be posted on a refereed preprint. Other results will be reported in aggregate to report on Review Commons.
Will the editorial decision of the affiliate journals be visible to each other?
No, the decisions of the affiliate journal will not be visible to other affiliate journals during transfer. If authors are unsatisfied with their reviews, they can withdraw from Review Commons; in this case reviews won’t be posted to bioRxiv and authors can proceed to submit to another journal to start review afresh. As described in the Guidelines for Authors, “Review Commons reserves the right to reject manuscripts at any point if ethical, biosecurity or scientific integrity issues arise. In serious cases, Review Commons may post a note of editorial concern next to the public reviews.”
Future perspectives for Review Commons
How will Review Commons scale up?
Currently there are 16 EMBO scientific editors accessing a network of 20,000 registered referees, which would otherwise take several years to build from scratch. In the future, EMBO may consider deploying additional editors from affiliate journals, increasing the scope of research reviewed, and including other journals in the affiliate network.
Would Review Commons consider partnering with community review projects?
Perhaps, but probably not in the beginning due to limited resources. Efforts are underway to make Review Commons more interoperable and portable for potential integration with other peer review projects.
Public perception and cultural change in publishing
Will journals view Review Commons Refereed Preprints as publication?
Review Commons will not make an editorial accept/reject decision, which is one of the features distinguishing it from journals. That said, it’s unclear whether some journals outside of the affiliate network will view this as competition in publishing.
Who will consider a Review Commons Refereed Preprints as valid as journal publication?
It will depend on the different contexts of ‘validity.’ Valid for readers keeping up with the latest science? Valid for career assessment? Valid for funding? It will be interesting to see how Refereed Preprints are adopted across these use cases.
By Victoria Yan