Chris Baur, Marek Łaska, Chrissy Prater, Michele Avissar-Whiting (Research Square)
Website or social media links
Current stage of development
Background information on current practices
Preprints are often shared with the specific intention of drawing scrutiny and feedback from the scientific community, yet engagement in the form of substantive comments or reviews has been low.
Overview of the challenge to overcome
The institution of academic peer review is dominated by journals that use editorial moderation to perform peer review in the context of journal scope and impact. Reviewer capacity is already heavily strained by the volume of submissions needing review at these journals. Given that a number of incentive structures already exist to encourage reviewers to volunteer their time to traditional journal peer review, how can the prospect of preprint review compete for this limited resource?
The ideal outcome or output of the project
A marked increase in reviews conducted on preprints and commensurate use or consideration of these reviews by journal editors.
Description of the intervention
Leverage the fact that depositing a preprint is free to encourage authors to ‘pay it forward’ by providing a preprint review for someone in their field who
a) is not currently under review at a journal,
b) has specifically indicated interest in having their preprint reviewed and
c) (optionally) has accrued interest in their preprint from others in the community (some simple counter could be implemented)
The angle here is that people who are provided a free service with some inherent value should be eager to espouse the values that the service represents. In this case, preprints represent a shift toward lowered barriers and greater transparency in the sharing of research, and preprint review represents an important development toward post publication peer review. There is a community of researchers who see great value in this proposition and would understand that the shift they long for will not actualize without their participation. Recent data has shown high levels of support for preprints, particularly among younger scientists. Partnering with institutional journal clubs or postdoc groups could be a way to engage an already bought-in cohort who would also benefit from the experience, start building their reviewer profile early, and get plugged into a range of peer review mentorship resources. We should start with them and encourage their early adoption, framed as participation in a grassroots movement.
Further development of this idea would capitalize on the concept of ‘status’, wherein regular contributions of reviews that meet a certain standard (perhaps those defined by PREreview) pay off in the form of recognition, for example a series of badges or visual indicators of contribution. This is, of course, the basis of the Publons project; however, we believe that Publons’ distance from the source has limited its impact. This recognition system would “live” in the same space as the review and its preprint.
Plan for monitoring project outcome
- Tracking engagement from reviewers and journals during a 6-month trial period
- Tracking review-related metrics (TAT, length, quality)
- Collection of qualitative feedback via surveys
What’s needed for success
Additional technology development
- Messaging at the point of preprint submission
- Curation and surfacing of preprints needing review, by subject matter/expertise
- Design and implementation of a platform-agnostic tier system for reviewers (badging/tagging)
- High-profile sponsorship of the initiative and comms
Feedback, beta testing, collaboration, endorsement
This project will require collaboration among preprint platforms that wish to be involved in the trial and at least one organization that has already established a system for preprint review. Mechanisms for feedback and beta testing will be agreed in the course of project planning.
Product management/Design: ~15k
Project management: ~15k