As part of the July 21, 2021 FeedbackASAP meeting, Ludo Waltman (CWTS, Leiden University), James Fraser (UCSF), Cooper Smout (Free Our Knowledge) organized a breakout session entitled “Posting journal reviews on preprints” to identity an evolutionary bridge between a system based around journal-organized peer review and referred preprints.
Peer reviewing: for whom and why?
The organizers opened the session with some prompts to gauge participants’ experiences with preprinting and peer review. In general, attendees were positive about posting preprints and the experience of receiving feedback on them, but were more ambivalent about performing review on behalf of journals. While participants enjoyed providing feedback to authors, some expressed philosophical discomfort with acting as a gatekeeper. This prompted a discussion, led by Mike Eisen (eLife), about the many roles of journal peer review, one of which is to serve “never readers:” decision-makers and evaluators who will eventually care that the paper was accepted by the journal in question, but never actually read it. Preprint review, instead, serves authors and readers more directly. Free of the need to act as a gatekeeper, reviewers of preprints can write collegial and helpful reviews they’d be happy to share publicly.
Despite the fact that many people are dissatisfied with journal review, few people are reviewing preprints, perhaps because, as one participant suggested, there are no concrete rewards or consequences for doing so. This session aimed to use the existing practice of journal review to accelerate preprint review, creating a more constructive review culture along the way.
Ludo noted that changes in publication and peer review practices are often slow, perhaps because modifying the system requires the coordination of many different stakeholders: editors, editorial boards, publishers, reviewers, authors, and so on. Arguing that we can speed up innovation via grassroots action that requires little coordination, he proposed to launch a campaign following the lead of session co-organizer and ASAPbio Vice President James Fraser (UCSF).
James Fraser’s preprint reviewing policies
James explained that he has adopted a policy to only accept journal review invitations when the manuscript in question has been posted as preprint; he then posts the review completed for the journal as a comment on the preprint. If there is no preprint, James refuses the invitation to review, citing his policy, and cc’s authors. This strategy has prompted at least one group of coauthors to post their work as a preprint. In order to facilitate this action, both James Fraser and Mike Eisen indicated they’d be willing to help colleagues maintain their anonymity by posting reviews on their behalf.
James has also taken this idea one step further: when asked to guest edit a paper for a journal, he instead organized an independent review process, inviting his own reviewers. With the consent of all reviewers, he then posted the entire package of reviews as a comment on bioRxiv.
Proposals for campaigns
Ludo set forth a few proposals for pledges or campaigns. The discussion focused on two pledges:
- As a reviewer for a journal, when I review an article that is already available as a preprint, I will post my review publicly
- As a reviewer for a journal, I will only review an article if it is already available as a preprint, and I will post my review publicly
Participants raised concerns about how early career researchers (ECRs) might be disproportionately affected by taking these actions: ECRs may be hesitant to post reviews on preprints for fear of offending either authors or journal editors. A number of solutions to this were discussed, for example posting reviews anonymously, pseudoanonymously, or under the name of more established researchers who are less prone to these concerns (e.g., James or Michael). Despite these concerns, many session participants indicated a willingness to sign on to one or both of these campaigns, and organizers have begun developing these ideas into a collective action campaign that will be posted on the Free Our Knowledge website in the coming weeks. In the meantime, interested members of the community are invited to provide feedback and indicate their preliminary interest in the campaign, either by posting comments on the GitHub thread directly or contacting Cooper Smout and Ludo Waltman.
