Joseph Wade, Wadsworth Center, New York State Department of Health
Website or social media links
Current stage of development
Ready to be implemented
One year initially, but potentially indefinitely.
Background information on current practices
The current system of peer review is highly inefficient. Many papers are rejected from at least one journal, and are re-reviewed at each journal. Reviews and the authors’ responses are rarely made public. A few journals allow authors to transfer reviews from previous submissions, and in the case of eLife, authors can have reviews attached to a preprint. However, there is no widely adopted system to review preprints.
The goal is to create a widely adopted system for peer review of preprints. The key part is “widely adopted”, since preprint review processes already exist but are rarely used.
The ideal outcome or output of the project
A preprint review portal that is directly linked from preprint servers during the preprint submission process. Authors would suggest up to five possible editors, who would be contacted by email. If an editor agrees to handle a paper, their role would be to recruit three reviewers and to ensure that those reviewers post a review through the review portal. Authors then have the opportunity to respond to the reviews and to submit a revised manuscript to the preprint server.
Create a website that authors are directed to (with the option to decline) when uploading a preprint. The authors would then provide a list of possible “editors”, who would be emailed automatically to ask if they would be willing to handle the paper. The first editor to accept would be chosen. The editor would be responsible for identifying peer reviewers, and soliciting 3+ reviews. Reviewers would be selected through the website by the editor, and would submit their reviews through the website. The editor would accept the reviews, or reject them in rare cases where they are inappropriate. The reviews would then be available for the authors to read – the editor’s role is simply to solicit reviews, not assess them, although there could be an option for the editor to write a 2-3 sentence “impact summary”. Reviews would be posted to the preprint server alongside the paper, and the authors would have the option to upload a response letter with a revised version of the paper.
In the initial period (1 year?), there would be no incentive for editors and reviewers. However, after the initial period there would be a credit system. All authors are given a certain number of free review credits, e.g. credit for reviews of three papers. Without credits, a review could not be requested. Editors and reviewers would gain credits for their service. This is a key difference from services like Review Commons that require professional editors, and presumably will require payment.
- How many authors selected the review process?
- How many editors/reviewers accepted?
- What fraction of papers received three reviews?
- Were the authors satisfied with the process? ( would require a follow-up survey that could be sent along with the reviews)
A web-based portal for preprint peer review is essential, and this needs to be integrated within the existing preprint websites to maximize author buy-in. It could simply be a link to an external site, with a system to post editor-approved reviews on the preprint site, alongside the associated papers. There are currently sites that facilitate preprint peer review, but they are rarely used, and are not linked to the preprint upload process.
Ideally, the project would get buy-in from journals, since I anticipate that many authors will want to transfer the reviews to a journal submission. Ultimately, I foresee a world where people only publish on preprint servers, but initially, journals would need to agree to use peer reviews acquired through the preprint peer review pipeline.
Paying authors, reviewers, and editors, would certainly promote buy-in, and would be fairly cheap. The system would require a website that would need to be designed and maintained.