Felix Richter (Kennedy Institute of Rheumatology, University of Oxford)
Nicolas Vabret (Icahn School of Medicine at Mount Sinai, New York)
1 Vabret N, Samstein R, Fernandez N, Merad M; Sinai Immunology Review Project; Trainees; Faculty. Advancing scientific knowledge in times of pandemics. Nat Rev Immunol. 2020 Jun;20(6):338. doi: 10.1038/s41577-020-0319-0. PMID: 32327718; PMCID: PMC7187143.
Current stage of development
Ready to be implemented
One year (01/2021-01/2022)
Since the Kickoff, we continued to develop our joint journal format. We have settled for an online voting system to systemically evaluate presented preprints, so that every voice among the journal club audience is equally represented and participation is increased. We have had our first joint journal club session with 70 attendees equally split between our institutions. Using the feedback for mentoring/teaching early career researchers in critical reviewing of preprints, we have reached out to PREreview for further advice and hopefully collaboration. We expect that a collaboration will allow us to train our members through “PREreview Open Reviewers”, an established mentoring program. In return, we hope to serve as a model system to inspire new journal club hubs around the world that would be linked to their platform and help reviewing preprints on a large scale. As a result, we expect to improve the legitimacy of our approach by improving our members’ training, diversity and representation.
How has your project changed?
The expected addition of PREreview to our initiative would increase the educational value of our initiative, the reviewing expertise and skills of our early career researcher members and even allow them to qualify for reviews assigned by journal editors. In the operational aspect of the project, we remain determined to keep this a grass-root initiative as previously described.
Have you integrated any feedback received?
Absolutely, we’ve benefited from the kickoff meeting to get new ideas to improve our legitimacy through more diverse representation. We also learned from the PREreview approach on their aspects of training early career scientists in peer review. This was one of the main issues we were keen to get further expertise on and are happy to potentially have found a partner to collaborate with on the issue.
Have you started any collaborations?
While the details are not fully established, we expect to have started our collaboration with PREreview in the coming weeks and are very hopeful and excited to get them involved.
Background information on current practices
During the COVID-19 pandemic, literature initiatives, as seen at Mount Sinai and the University of Oxford, have taken advantage of their journal club communities to evaluate and critically analyse a large number of SARS-CoV-2 preprints. These efforts led these two communities to write and share public reviews of selected preprints to provide independent insights to frontline researchers, to the general public, and to further help prevent misinformation during this pandemic1. This community effort brought together students, post-doc and faculty members into a common goal of improving rapid communication of scientific findings (incl. ‘live reviews ’).
Overview of the challenge to overcome
- Large numbers of preprints are difficult to assess with a limited pool of reviewers.
- Early career scientists have limited access to the reviewing process and opportunities to develop their critical thinking.
- Large numbers of preprints make it difficult to identify field-specific research trends.
The ideal outcome or output of the project
- Repurpose institutional journal clubs to take part in evaluating up-and-coming research on different research fields and preprint servers.
- Training ground for research trainees to learn how to critically assess scientific literature.
- Improve the scientific legitimacy and reduce the inherent bias of a system constraint around a single institution and promote open science.
Description of the intervention
Mount Sinai and the University of Oxford have yielded fast and efficient identification of the latest developments in SARS-CoV-2 research, we now sought to formally join our journal clubs to assess more generally preprints in the field of immunology. Using virtual platforms, we will organize joint Journal Club meetings to discuss the latest preprints, which will be screened by and evaluated by early career scientists and discussed in the joint meeting. Taking advantage of an ongoing collaboration with Nature Reviews Immunology (NRI), we will select two preprints each month to further highlight in the journal. To provide the public simple access to our reviews, we will create a website which will contain links to all assessed preprints, their reviews and summaries. By creating dedicated social media accounts, we will further improve the visibility of preprint reviews and increase discussion on the latest preprints with the scientific community.
We aim to serve as a blueprint for other institutions in the future and foster a community-driven approach to improve the quality of science.
Plan for monitoring project outcome
- Feedback from surveys of each institution involved Journal Club and the authors of the preprints.
- Expand involved initiative concept to other institutions.
- Measure impact of highlighted preprints through monitoring community discussion and NRI access numbers.
What’s needed for success
Additional technology development
A dedicated platform where immunology-focussed preprints would be automatically referred would be a helpful tool to ensure systematic screening.
Feedback, beta testing, collaboration, endorsement
- Collaboration with established publishers are critical to improve visibility of preprints and their reviews in open access format.
- Support of our research communities, especially senior research members to provide early career researchers a formal training in article reviewing.
- Additional platform that helps advertising our approach to increase awareness of this cross-institutional initiative.
- Resources to cover admin costs (online maintenance, communication between initiative parts).
- Costs for an editor-led workshop on critical assessment of scientific preprints for early career scientists.