In our 6-month trial of the Preprint Reviewer Recruitment Network, participating journals made 124 total additions to their reviewer databases; they also extended 28 invitations to review and 62 invitations to advisory or editorial boards. We plan to extend the program into a second phase.
Many early-career researchers want to get involved with journal peer review, but they don’t have a clear way to get their foot in the door. At the same time, facing challenges in recruiting reviewers and seeking to correct disparities in representation, journal editors are looking to broaden their reviewer pool, but this can be an onerous task among their many editorial duties.
Preprints offer a solution: researchers can post their own public comments or reviews on preprints, creating a work sample that lets editors assess their ability to write a constructive review report. Importantly, these reviews also benefit both other readers and the authors of the preprint, who can incorporate the feedback into an improved version of their paper.
To help bridge the gap between journals seeking new reviewers and editors and researchers already reviewing preprints, we created the Preprint Reviewer Recruitment Network. Over a 6-month trial period, we invited researchers to fill out a form containing their contact information, profile (such as field, institution, link to CV, and career stage) and links to either preprint comments or reviews they had authored. All in all, we received over 300 entries into this database. We partnered with 46 journals and one review platform, all of which agreed to consider these researchers for possible addition to their reviewer or editorial pool.
Supporting new preprint reviewers
One of the first things that we noticed was that many people who signed up didn’t actually include any preprint reviews. Even by the end of the assessment period, only about 15% of entries to the database contained these clear links. So, in order to support network members in providing public preprint feedback, we created a support system, including a Slack channel where we encouraged researchers to partner up to write reviews. We also organized events such as workshops on how to peer review, a live preprint journal club in collaboration with PREreview, and co-working sessions during which members could optionally collaborate.
Overcoming the energy barrier to performing preprint review
While dozens of network members attended the events and joined the Slack channel, we didn’t see a large number of people updating their network profiles. Why was this so difficult? In a survey of network members with 56 respondents, the most common reason given for not yet completing a public preprint review was “lack of time.”
Note that these are the same researchers who wish to receive an invitation from journal editors to perform preprint review, suggesting that they might be willing to make time for peer review if actively invited.
Aside from the time issue, researchers also mentioned a desire to develop review skills. It’s possible that we could do more to provide support, but we’re also conscious that many institutions, publishers, and organizations such as PREreview are already offering great peer review training programs.
Enthusiastic response from journals
While some editors found it difficult to integrate the Network into current workflows, overall, journals responded enthusiastically to the 47 network entries with preprints reviews.
By the end of the trial, journals made 124 total additions to their reviewer databases; they also extended 28 invitations to review and 62 invitations to advisory or editorial boards.
Researcher experience
“After joining the preprint reviewer recruitment network, I was contacted by two journals (eLife & FEBS Open Bio), who encouraged me to join their pool of volunteer peer-reviewers. I believe the preprint reviewer network is a great platform to support (early-stage) researchers to forge new connections and bring diversity to the global peer-reviewer pool.”
– Sree Rama Chaitanya Sridhara, postdoctoral research associate, CDFD, India
Designing phase 2
We’re encouraged by these results, and will take what we’ve learned from the pilot into a second phase of the Network.
First, we’ll ask new members of the Network to provide a brief biography that provides insight into their expertise and research interests. We’ll also collect a bit more structured information on their publication history and links to external resources, like ORCiD.
Second, we will encourage researchers to only sign up once they have public preprint reviews and comments to share. This will streamline the journal’s evaluation of potential candidates.
Third, we will implement quarterly deadlines for researchers to update their profiles. After this deadline, we’ll contact journals with a user-friendly list of suitable reviewers. The first deadline will be June 15, 2022.
If you’re a researcher interested in sharing your preprint reviewing experience with journals, please sign up here! And, if you’re an editor who would like access to the Network for the purpose of recruiting reviewers, editors, or advisors, please email jessica.polka@asapbio.org.