2021-10-18 correction: In the original version of this post, the data labels for “accepted” and “rejected” re-review rates were swapped. The correct labels now appear below.
In late 2019, EMBO and ASAPbio launched Review Commons, a platform for journal-independent peer review that facilitates the posting of a refereed preprint and submission to 17 partner journals. With this system, we aimed to improve peer review in three ways:
- Focus peer review on the science. When evaluating a paper for a journal, reviewers may be tempted to act as gatekeepers who evaluate the fit of the paper relative to the journal’s editorial standards. Instead, Review Commons explicitly asks reviewers to focus on evaluating and improving the science for its own sake, without a journal in mind.
- Increase efficiency by eliminating cycles of re-reviewing. About half of biomedical articles are submitted to more than one journal before publication. Rejection of papers after peer review contributes to an estimated 15 million hours of wasted reviewer time across all fields per year. Review Commons partners with 17 affiliate journals who agree to consider papers without repeating peer review.
- Accelerate transparency. Right now, most journal peer review functions as a black box. Many journals are beginning to offer authors the option to publish peer review reports and the journals affiliated with Review Commons have all implemented a form of transparent review. As preprints become more and more common, there’s now an opportunity to accelerate transparency even further by sharing peer reviews prior to journal submission, enabling reviews to inform readers while the work is under revision. Review Commons authors can elect to post a refereed preprint, including review reports and their rebuttal. These are visible from a “peer review” tab on bioRxiv and are available in a machine-readable form through the API of the hypothes.is platform used to host and publish the reviews.
Review Commons is an experiment, and as such, we’re committed to measuring and evaluating the outcomes of the project. Here’s what we learned in the first 9 months.
Science-focused peer review
In surveying corresponding authors who used the service through June 30, 2020, many authors found the reviews more collegial, reasonable, and unbiased than those received in traditional review processes. Note that many authors felt uncertain about the overall speed and efficiency of the publishing process, likely due to the fact that the majority of their papers were still in process.
A major challenge in implementing a high-quality science-focused peer reviews process is to obtain referee reports that are usable by a broad variety of journals. To this date, 16 journals of the consortium have accepted papers based on the reports forwarded by Review Commons, serving as a positive indicator of their utility and broad portability.
Efficiency
Over the evaluation period, it took Review Commons a median of 37 days to return peer reviews to authors whose manuscripts were sent for peer review. Authors spent a median of 27 days revising their manuscripts and responding to reviewer comments, and affiliate journals returned a first decision in 8 days and accepted manuscripts, on average, 11 days after that (note that this figure includes papers accepted at first decision). Review Commons manuscripts are transferred to a median of two affiliate journals.
The system is effective at avoiding duplicate peer review, with affiliates not seeking additional peer review in 98% of papers they reject and 86% of papers they accept. We suspect that this will lead to reductions in the overall time to publication. However, before drawing firm conclusions, we need to wait for more manuscripts to go through the system so as not to truncate the distribution.
Transparency
90% of the papers published after review by Review Commons are accompanied by transparent peer review, meaning that the Review Commons process is being disclosed along with the final version of the article. Review Commons also gives authors the option to disclose these reviews earlier through the posting of a refereed preprint. Among authors who choose this option, all of them cite the desire to “support open science” as important to their decision. However, only 30% of the manuscripts that received reviews from Review Commons have been posted as refereed preprints. Why aren’t more authors posting their reviews?
Because the service is young, a significant fraction (approximately ¼) manuscripts are still waiting for authors to respond to the reviews, meaning that the fraction of posted preprints may rise. Nevertheless, we surveyed on why they did or didn’t elect to post a refereed preprint. All authors who posted a refereed preprint cited support for open science as an important reason for doing so. The most popular reason cited for not posting a refereed preprint was the need for more time to revise the manuscript. On the whole, however, ⅓ to ½ of authors listed “don’t know” as their response to each facet of this survey question. Since receiving these results, we’ve implemented an opportunity for authors to consult with the Managing Editor after receiving their reviews. Anecdotally, these discussions seem to help authors understand the potential benefits of posting a refereed preprint.
Have more questions? Take a look at the recording of our September 8, 2020 webinar, see the slides from the event, or visit the FAQ. You can also follow Review Commons on Twitter, and check out the latest Refereed Preprints. Contact us directly at contact@reviewcommons.org!
See the images from this post in an infographic.
