Felix Richter (Kennedy Institute of Rheumatology, University of Oxford)
Nicolas Vabret (Icahn School of Medicine at Mount Sinai, New York)
1 Vabret N, Samstein R, Fernandez N, Merad M; Sinai Immunology Review Project; Trainees; Faculty. Advancing scientific knowledge in times of pandemics. Nat Rev Immunol. 2020 Jun;20(6):338. doi: 10.1038/s41577-020-0319-0. PMID: 32327718; PMCID: PMC7187143.
Current stage of development
Ready to be implemented
One year (01/2021-01/2022)
Background information on current practices
During the COVID-19 pandemic, literature initiatives, as seen at Mount Sinai and the University of Oxford, have taken advantage of their journal club communities to evaluate and critically analyse a large number of SARS-CoV-2 preprints. These efforts led these two communities to write and share public reviews of selected preprints to provide independent insights to frontline researchers, to the general public, and to further help prevent misinformation during this pandemic1. This community effort brought together students, post-doc and faculty members into a common goal of improving rapid communication of scientific findings (incl. ‘live reviews ’).
Overview of the challenge to overcome
- Large numbers of preprints are difficult to assess with a limited pool of reviewers.
- Early career scientists have limited access to the reviewing process and opportunities to develop their critical thinking.
- Large numbers of preprints make it difficult to identify field-specific research trends.
The ideal outcome or output of the project
- Repurpose institutional journal clubs to take part in evaluating up-and-coming research on different research fields and preprint servers.
- Training ground for research trainees to learn how to critically assess scientific literature.
- Improve the scientific legitimacy and reduce the inherent bias of a system constraint around a single institution and promote open science.
Description of the intervention
Mount Sinai and the University of Oxford have yielded fast and efficient identification of the latest developments in SARS-CoV-2 research, we now sought to formally join our journal clubs to assess more generally preprints in the field of immunology. Using virtual platforms, we will organize joint Journal Club meetings to discuss the latest preprints, which will be screened by and evaluated by early career scientists and discussed in the joint meeting. Taking advantage of an ongoing collaboration with Nature Reviews Immunology (NRI), we will select two preprints each month to further highlight in the journal. To provide the public simple access to our reviews, we will create a website which will contain links to all assessed preprints, their reviews and summaries. By creating dedicated social media accounts, we will further improve the visibility of preprint reviews and increase discussion on the latest preprints with the scientific community.
We aim to serve as a blueprint for other institutions in the future and foster a community-driven approach to improve the quality of science.
Plan for monitoring project outcome
- Feedback from surveys of each institution involved Journal Club and the authors of the preprints.
- Expand involved initiative concept to other institutions.
- Measure impact of highlighted preprints through monitoring community discussion and NRI access numbers.
What’s needed for success
Additional technology development
A dedicated platform where immunology-focussed preprints would be automatically referred would be a helpful tool to ensure systematic screening.
Feedback, beta testing, collaboration, endorsement
- Collaboration with established publishers are critical to improve visibility of preprints and their reviews in open access format.
- Support of our research communities, especially senior research members to provide early career researchers a formal training in article reviewing.
- Additional platform that helps advertising our approach to increase awareness of this cross-institutional initiative.
- Resources to cover admin costs (online maintenance, communication between initiative parts).
- Costs for an editor-led workshop on critical assessment of scientific preprints for early career scientists.