PREreview: Antoinette Foster, Monica Granados, Samantha Hindle, Katrina Murphy, Daniela Saderi
Simply Secure: Georgia Bullen, Carissa Yao
Website or social media links
Current stage of development
Ongoing / idea
How has your project changed?
The Sprint confirmed that our project roadmap is developing along the right lines to meet the needs of the community, and has been further strengthened by expanding our collaborations (see below).
Have you integrated any feedback received?
The Sprint highlighted several common themes across the proposal—mainly the need to filter content and rapidly surface preprints of high importance, to fix the problems in the peer review system, and to create a more equitable scholarly landscape. We heard from participants in our breakout session that our focus on Diversity, Equity, and Inclusion (DEI) and training to expand the peer reviewer pool was what drew them to our project, and would be particularly valuable to the ecosystem. This feedback confirmed our approach for upcoming platform features and community engagement programs, specifically the ability to write both long and short-form/templated rapid reviews on PREreview across disciplines, the development of community pages that will allow preprint journal clubs groups to create and manage their own groups on the PREreview platform, and the growth of our training programs to meet the needs of early-career researchers, institutions, and editors who are looking to build the next generation of racially diverse, socially-conscious, and constructive peer reviewers.
Have you started any collaborations?
Yes! Collaboration is at the core of our values, and is necessary for us to build a more equitable scholarly environment that transcends the “one-solution-fits-all approach”. The Sprint allowed us to strengthen previously established collaborations and identify new partners with which we can work together to achieve our shared goals:
- We will join forces with Train Centre in Communication (TCC) Africa and AfricArXiv to co-develop a peer review mentoring program adapting the work we have done for our Open Reviewers pilot to meet the needsand cultural context of the African research community. Together we will also organize and co-facilitate live-streamed preprint journal clubs featuring preprints authored by African scientists who request constructive community feedback.
- We also aim to share some of the resources we developed for the Open Reviewers program—such as our bias reflection guide and peer review assessment guide—with Life Science Editors to maximize their impact and potentially be the foundation for a similar program aimed at training Editors.
- At the breakout session run by Felix Richter and Nicolas Vabret , we realized the importance of platform features we have been developing that will support different models of community review—for example, the ability to share review drafts with mentors, a model we have been developing for our Open Reviewers program, and can be incorporated into preprint JCs for feedback before review publication. We will be collaborating with Felix and Nicolas to build this feature to facilitate their efforts of leveraging institutional journal clubs to review preprints.
- We connected with Michele Avissar-Whiting (Research Square) who intends to integrate the PREreview platform into their ‘pay it forward’ proposal for scholarly peer review.
- Last but not least, we will explore ways to collaborate with Stencila on the development and use of collaborative writing components to add to the PREreview platform, thereby simplifying the collaborative review experience for our community.
Research historically is assessed by a handful of “expert” reviewers, rarely selected from a diverse pool, limiting the comprehensive and context-appropriate evaluation of the work.
Preprints offer opportunities to rethink how, when, and where scholarly output is disseminated and by whom it is evaluated. Traditional definitions of reviewer expertise need to be redefined through an equity perspective to include systematically excluded researchers. We need a concerted effort to both dismantle the current biases and build a new cultural ethos around peer review.
At PREreview we aim to 1) increase diversity in peer review by training the next generation of equitable and socially conscious reviewers, 2) encourage collaborative discussions through live-preprint journal clubs (JCs), and 3) develop tools that allow researchers to request or write preprint reviews.
Incentives and outputs
We will build pathways for early-career researchers (ECRs) to learn together and raise their profiles as expert reviewers through community engagement with the following initiatives:
- PREreview Open Reviewers Programs:
- Opportunities for free mentorship and mentorship training
- Increased visibility as expert reviewers
- Trained reviewers receive badges upon stage completion (Figure 1)
- Database of diverse trained reviewers for editorial use
- Program materials (CC-BY)
- Metrics of success:
- Number of trained reviewers/mentors
- Number of participating publishers using the database
- Surveys assessing mentees growth and reviewer confidence
- Live-streamed preprint JC facilitation training:
- Free facilitation training to reduce barriers for researchers to organize/run JCs in their communities
- Trained JC facilitators receive badges and are added to a database amplifying the number of global preprint discussions
- Toolkit and training video (CC-BY)
- Metrics of success:
- Number of trained facilitators, JCs, and collaborative reviews generated
The ideal outcome or output of the project
Description of the intervention
PREreview Open Reviewers is a pilot peer review mentoring program that trains ECRs in peer review, combating personal bias, and establishes 1:1 mentorship with journal editors. The program has in-built capacity to train hundreds of ECRs as expert reviewers with skills to train and empower the next generation of peer reviewers:
Figure 1: Program stages
To scale and engage globally diverse researchers, we will seek collaborations with organizations serving diverse researcher communities to adapt the program to their needs.
Live-streamed preprint JCs facilitate structured and constructive preprint discussions, incorporating diverse methods and global participation, the fruits of which often contribute to the editorial peer review process.
What’s needed for success
Additional technology development
- Develop platform features to include badging and authored-driven solicitation of a live JC or mentorship
- Develop a reviewer database integrated into established editorial databases
Feedback, beta testing, collaboration, endorsement
- Feedback on building initiatives that support global and equitable participation
- Collaboration with other like-minded organizations to adapt our programming to their communities’ needs
- Collaboration and endorsement from publishers who use our database
The initiatives above are contingent on financial support via individual and/or collaborative grant funding, sponsorships, and/or partnerships.