Daniela Saderi (PREreview Co-Founder and Director) and Sarah Greaves (STM Publishing Consultant)
Website or social media links
- OASPA announcement
- Project website
- Digital Science Blog Post: “Does creating a rapid reviewer pool improve trust in COVID-19 peer review?”
- Digital Science Blog Post: “Creating a rapid reviewer pool…..and trying to use it”
- Poster (Zenodo)
Current stage of development
Ready to be implemented
Background information on current practices
In May 2020 a group of STM publishing organisations made a direct call to the whole community to maximize the efficiency of peer review of COVID-19 research ( “COVID-19 Publishers’ Open Letter of Intent – Rapid Review”). The community responded quickly, with over 1700 researchers signed up to a rapid reviewer pool (volunteer rapid reviewers). All are committed to quickly reviewing COVID-19 manuscripts, to have their reports shared between publishers if manuscripts are transferred and for them to be openly published.
The group of organisations involved is now greater than 20 and wishes to focus on engaging the volunteer rapid reviewers in the review of COVID-19 preprints. Outbreak Science Rapid PREreview (OSrPRE), an independent preprint review platform, has been a member of the collaboration since May. To extend the aims of the group, OSrPRE has prototyped a COVID-19 preprint review dashboard. The dashboard allows readers to filter recommended reviews, show preprints ready for journal-organized peer review, and highlights those that have data and code (screenshot below). The group wishes to support this further and connect the rapid reviews with journal Editors to decrease assessment time and help triage incoming manuscripts.
- Train and support engagement of the volunteer rapid reviewers with preprints
- Maximise impact of preprints with Editors by: a) highlightCOVID-19 preprints that have reviews; b) identify COVID-19 preprints recommended for peer review; c) identify preprints that have open data/code; d) identify COVID-19 preprint reviewers directly to Editors.
The ideal outcome or output of the project
- A community engagement strategy to train, manage, and support the volunteer rapid reviewers group
- Seamless workflow connecting reviews of COVID-19 preprints with Editors
- Surveys for editorial teams and volunteer rapid reviewers
Description of the intervention
An engagement strategy includes:
- Inviting volunteer rapid reviewers to review COVID-19 preprints on OSrPRE
- Organizing live-streamed preprint journal clubs and webinars to train the volunteer rapid reviewers
- Offering badges for volunteer rapid reviewers to be recognized as active contributors
Easy interventions are needed to connect Editors and content/reviewers including:
- A demo guide and 1:1 support for Editors in how to use the current COVID-19 dashboard
- Weekly summary reports and ongoing support
An API integration between OSrPRE and the journal submission software would:
- Allow authors and/or editors to send request calls for rapid reviews of submitted COVID-19 manuscripts
- Allow authors and/or editors to be notified when new/requested rapid reviews are published
- Allow authors to recognize if an OSrPRE user is a volunteer rapid reviewer and connect with them
We will design the following surveys:
- Editorial teams in the collaboration to understand their needs
- Volunteer rapid reviewers active on OSrPRE
To measure the impact of our efforts we will:
- Track engagement on the dashboard
- Count the number of rapid reviews of COVID-19 preprints
- Count the number of journal editors who report to regularly use the dashboard
- Count the number of rapid review requests from journal editors/manuscript authors
- Count the number of volunteer rapid reviewers who are active on OSrPRE
- Measure the extent and impact of our engagement strategy via opt-in surveys
What’s needed for success
Additional technology development
API integrations — OSrPRE has already a basic API that would need to be customized for the purposes described above.
Feedback, beta testing, collaboration, endorsement
We welcome feedback, beta testing on the dashboard and integration, collaboration, endorsement.
Some of the more technologically-heavy interventions described above are contingent on funding (up to $50,000 to cover for a part-time project manager and development).