Welcome to the new ASAPbio website! See what’s on the roadmap for 2025.
- This event has passed, but you can find documentation and materials below.
#FeedbackASAP (2021)
July 21, 2021 @ 3:00 pm - 7:00 pm UTC+0
Public feedback on preprints can unlock their full potential to accelerate science.
Public preprint review can help authors improve their paper, find new collaborators, and gain visibility. It also helps readers find interesting and relevant papers and contextualize them with the reactions of experts in the field. Never has this been more apparent than in COVID-19, where rapid communication and expert commentary have both been in high demand. Yet, most feedback on preprints is currently exchanged privately.
On July 21, 2021, ASAPbio, in partnership with DORA, HHMI, and the Chan Zuckerberg Initiative, held a meeting to discuss how to create a culture of constructive public review and feedback on preprints.
Agenda
| Time in UTC (duration) | Title | Description |
| 15:00 (5’) | Welcome | Kickoff and announcements |
| 15:05 (30’) | Why preprint feedback? | Researchers from a variety of career stages and disciplines will discuss the benefits of creating a culture of open feedback on preprints.
|
| 15:35 (25’) | Institutional support for preprint feedback | Leaders of programs supporting preprint review at institutions & societies will discuss their motivations, experiences, and provide advice on how to get involved.
|
| 16:00 (20’) | What can you do? | Highlights of new initiatives you can join to help grow preprint feedback. |
| 16:20 (5’) | Introduction to breakout sessions | Overview of the sessions, how to join them, and instructions for the remainder of the meeting |
| 16:25 (60’) | Breakout session 1 | 1A: How to foster a positive preprint feedback culture: From FAST principles to implementation – Iratxe Puebla (ASAPbio)1B: Developing a taxonomy to describe preprint review processes – Victoria Yan (ASAPbio)
1C: Public preprint review as a tool to empower the next generation of socially-conscious peer reviewers – Mugdha Sathe (UW), Rebeccah Lijek (Mt. Holyoke), Daniela Saderi (PREreview) |
| 17:25 (10’) | Break | |
| 17:35 (60’) | Breakout session 2 | 2A: Posting journal reviews on preprints – Ludo Waltman (CWTS Leiden), James Fraser (UCSF), Cooper Smout (Free Our Knowledge / IGDORE)2B: Curation and review in the preprint landscape – Hannah Drury, Paul Shannon, Godwyns Onwuchekwa (eLife/Sciety)
2C: Tackling information overload: identifying relevant preprints and reviewers – Christine Ferguson, Martin Fenner, Daniel Mietchen |
| 18:35 (20’) | Report out | Short summaries from each breakout session |
| 18:55 (5’) | Wrapup and conclusion | Next steps and recap of major themes |
| 19:00 | Adjourn |
Take action to support preprint review
There’s a lot you can do right now to give and receive constructive feedback on preprints. We’ve collected some actionable ideas below.
As an author
Invite feedback on your preprint
Why request public feedback on your preprint?
By explicitly requesting public feedback on your preprint, you’ll help readers feel welcome to share their comments. Specifying the time frame and type of feedback you need can ensure the input you get is more likely to be constructive.
How to do it
Tweet out a request, or leave a comment on your own preprint inviting public review. Here’s a suggested format:
“My co-authors and I welcome public feedback on our preprint, ideally by [DATE]. We are especially interested in [statistics, etc].”
If you’re interested in finding these requests within the bioRxiv comment section, you can use see Vincent Rubinetti’s code.
Other ways to request feedback
On PREreview, any researcher (including authors) can request feedback on any preprint with a DOI as well as provide feedback to preprints in the form of rapid and full PREreviews.
Submit to a journal or service that posts reviews on preprints
Starting in July 2021, eLife will exclusively review preprints and post public reviews on them.
You can use our registry, ReimagineReview, to identify projects that review preprints. The following listings represent those that allow authors to submit their own preprints for feedback.
Submit your manuscript to our crowd review trial
We are thrilled to announce that we are running a trial to test the crowd review approach for preprint review. We invite cell biologists with an interest in preprints and preprint feedback to join this trial.
Why is ASAPbio running this trial?
While there is increasing interest in review activities around preprints, the level of public commenting and reviewing on preprints remains low overall. To explore review modalities that may foster participation in preprint review, we are adapting the crowd review model pioneered by the journal Synlett to preprints, to learn whether this approach provides an engaging format for researchers to participate in preprint review, and a way to generate public reviews on preprints.
What does the trial involve?
The trial will involve bioRxiv preprints in cell biology which authors have requested feedback on. We will coordinate a group of researchers interested in contributing feedback (‘the crowd’) and will circulate a paper or two weekly to the crowd, inviting comments over the following week. Crowd members can then comment on the full paper or parts of the study. After the commenting period, a collective synthesis of the comments will be posted as a public review via bioRxiv’s Transparent Review in Preprints (TRiP) workflow.
The trial will run for three months starting in August. We hope that this format will encourage broad participation in preprint feedback, particularly by early career researchers who are often underrepresented in journal review.
Get involved!
If you are an author of a cell biology preprint, please signal your interest in receiving feedback on your work by adding a comment on the preprint or by posting on Twitter with the hashtag #FeedbackASAP.
Read more about the trial and check for updates at asapbio.org/crowd-preprint-review.
As an reader/reviewer
Post reviews you complete for a journal on the preprint
James Fraser (UCSF) has adopted a policy in his lab compact to post all reviews as comments on bioRxiv (emphasis ours):
“We believe that transparency in peer review is critical to improving the quality and professionalism of reviews. We only review manuscripts that have been posted on BioRxiv and we post our peer review comments publicly on Biorxiv as well. Peer review is also viewed as a training opportunity and all lab members who co-review with JF are credited for their contribution to the review. JF recognizes that he holds certain privileges that allow him to post peer review comments non-anonymously without fear of repercussion. Unfortunately, this is not the case for all researchers. JF is willing to post reviews on behalf of anyone, both in and outside of the Fraser lab, who wishes to remain anonymous. Guidelines for how to engage in respectful and constructive peer review can be found on the Peer Review in the Life Sciences course page.”
Contribute to a preprint review project
You can use our registry, ReimagineReview, to identify projects that review preprints. The following listings represent those that allow authors to submit their own preprints for feedback.
Highlight less visible preprints
Preprints have great potential to democratize access and production of knowledge. However, researchers tend to be most aware of work occurring within their own network; highlighting and sharing only these preprints can contribute to the Matthew effects and limit readers’ exposure to a sliver of available science. Looking outside of the obvious candidates when highlighting or amplifying preprints can help. You can:
- Search regional servers. Because some servers may overrepresent certain countries (Abdill et al., 2020), check regional preprint servers such as AfricArxiv and RINarxiv to find papers that may be less visible to your colleagues outside those regions, or include the names of specific countries in your search to find work from them.
- Publish and search preprints in various languages. An increasing number of preprint repositories accept submissions in more than one language. Make strategic searches by keywords in English and their translations to your mother tongue. Check if a preprint already has one or more translations (of the abstract) available. For more on the importance of multilingualism, see the Helsinki initiative.
- Find untweeted preprints. For example, when looking through bioRxiv for her #365preprints project (see below), Prachee Avasthi recommends selecting preprints that have yet to be tweeted.
Thanks to Dasapta Erwin Irawan, Jo Havemann, and Stefano Vianello for their input.
DIY preprint highlights
In addition to using established tools, you can also create your own preprint highlights in a variety of formats.
For example, Prachee Avasthi uses Twitter Spaces to host a short audio conversation about an interesting preprint every day.
Jonny Coates, Emma Wilson, and John Howard are starting a podcast for longer discussions.
(Note that we’re piloting a program to provide support for mission-oriented expenses as part of our community program)
Share your experience with journals via the Preprint Reviewer Recruitment Network
Preprint feedback has the potential to not only help authors and readers, but also to identify potential reviewers and editorial board members for journals. Unfortunately, finding preprint reviews authored by a particular individual and linking them to other useful information (such as disciplinary keywords) remains challenging. The Preprint Reviewer Recruitment Network addresses these issues by enabling willing researchers to share preprint feedback as work samples for review by participating journals.
We expect that the network will:
- Provide journals and publishers with access to pools of potential reviewers or editorial board members
- Help researchers, especially ECRs, break into reviewing or editing roles
- Recognize the efforts of researchers who participate in public reviewing and commenting on preprints
The Preprint Reviewer Recruitment Network will run as a 6-month long pilot from 2021-07-21 to 2022-01-21. Results, such as the number of reviewers who volunteered and the number of invitations extended by each journal, will be reported at asapbio.org.
Join our crowd review trial
We are thrilled to announce that we are running a trial to test the crowd review approach for preprint review. We invite cell biologists with an interest in preprints and preprint feedback to join this trial.
Why is ASAPbio running this trial?
While there is increasing interest in review activities around preprints, the level of public commenting and reviewing on preprints remains low overall. To explore review modalities that may foster participation in preprint review, we are adapting the crowd review model pioneered by the journal Synlett to preprints, to learn whether this approach provides an engaging format for researchers to participate in preprint review, and a way to generate public reviews on preprints.
What does the trial involve?
The trial will involve bioRxiv preprints in cell biology which authors have requested feedback on. We will coordinate a group of researchers interested in contributing feedback (‘the crowd’) and will circulate a paper or two weekly to the crowd, inviting comments over the following week. Crowd members can then comment on the full paper or parts of the study. After the commenting period, a collective synthesis of the comments will be posted as a public review via bioRxiv’s Transparent Review in Preprints (TRiP) workflow.
The trial will run for three months starting in August. We hope that this format will encourage broad participation in preprint feedback, particularly by early career researchers who are often underrepresented in journal review.
Get involved!
Join the crowd preprint review trial to develop your reviewing skills and help us build trust in preprints. We invite cell biologists to sign up for participation as crowd preprint reviewers.
As an journal
Post reviews on preprints
eLife has announced that from July 2021, it will exclusively review preprints and post public reviews on them. According to the press release,
“eLife is refocusing its editorial processes towards transforming preprints into ‘refereed preprints’ that include a public assessment of the work prepared by the journal’s reviewers and editors. This will involve providing updated instructions for its reviewers to capture what should and should not be included in public reviews, and modifying its editorial processes to focus on the production of these reviews. “Our plan is to work closely with our editors, reviewers, authors and readers to optimise the process by which public reviews are created and their utility to the diverse audiences we hope to reach,” Eisen says.”
The SciELO journal Educação em Revista enacted a similar policy in February of 2021.
Join the Preprint Reviewer Recruitment Network
Preprint feedback has the potential to not only help authors and readers, but also to identify potential reviewers and editorial board members for journals. Unfortunately, finding preprint reviews authored by a particular individual and linking them to other useful information (such as disciplinary keywords) remains challenging. The Preprint Reviewer Recruitment Network addresses these issues by enabling willing researchers to share preprint feedback as work samples for review by participating journals.
We expect that the network will:
- Provide journals and publishers with access to pools of potential reviewers or editorial board members
- Help researchers, especially ECRs, break into reviewing or editing roles
- Recognize the efforts of researchers who participate in public reviewing and commenting on preprints
The Preprint Reviewer Recruitment Network will run as a 6-month long pilot from 2021-07-21 to 2022-01-21. Results, such as the number of reviewers who volunteered and the number of invitations extended by each journal, will be reported at asapbio.org.
Related projects
Norms working group
The Working Group will produce 1) a set of recommended cultural norms for participation in preprint review, 2) a library of available resources relevant to constructive review of preprints, and 3) materials to support dissemination of the above. The working group is chaired by Iratxe Puebla.
- Maryrose Franko (Health Research Alliance)
- Sharon Ahmad (The Company of Biologists)
- Hilda Bastian (hildabastian.net)
- Sara Monaco (Review Commons / EMBO)
- Dyche Mullins (UCSF)
- Samantha Hindle (PREreview)
- Sandra Franco Iborra (New York Genome Center)
- Gautam Dey (EMBL)
- Shriyaa Mittal (Harvard Medical School/MGH)
- Tim Behrens (University of Oxford)
Taxonomy working group
In order to facilitate the display, filtering, and interpretation of varied forms of preprint feedback, the taxonomy working group will create a simple, shared vocabulary for describing the most valued characteristics of preprint review. The working group is chaired by Victoria Yan.
- Gabe Stein (KFG/Doc Maps)
- Gary McDowell (KFG/Doc Maps)
- Tony Ross-Hellauer (TU Graz/Doc Maps)
- Kathleen Shearer (COAR Notify)
- Philip Cohen (SocArxiv)
- Thomas Lemberger (EEB)
- Phil Hurst (Royal Society)
- Clare Stone (SSRN)
- Damian Pattinson (eLife)
- Richard Sever (bioRxiv, medRxiv)
Breakout session descriptions
1A: How to foster a positive preprint feedback culture: From FAST principles to implementation – Iratxe Puebla (ASAPbio)
In this interactive session, we will discuss and collect input on the FAST principles for preprint feedback. We will also brainstorm how we can put the FAST principles into practice to support a positive culture of preprint feedback, and explore their value for early career researchers.
1B: Developing a taxonomy to describe preprint review processes – Victoria Yan (ASAPbio)
ASAPbio’s taxonomy working group has proposed a draft vocabulary for describing the most valued characteristics of preprint review and feedback. In this session, we would like to test the intelligibility of the taxonomy terms, understand which terms are useful to you, and how they can be used in your preprint workflow through 3 interactive activities.
1C: Public preprint review as a tool to empower the next generation of socially-conscious peer reviewers – Mugdha Sathe (UW), Rebeccah Lijek (Mt. Holyoke), Daniela Saderi (PREreview)
In this session, participants will discuss the barriers and opportunities for early-career researchers—from undergraduates to grad students to postdocs—to engage in preprint peer review. We will challenge the very definition of expertise, share our experiences working with ECRs in preprint peer review through efforts at PREreview and in the undergraduate classroom, and solicit participants ideas on how to streamline innovative and inclusive approaches to empower ECRs in engaging in the peer review process.
2A: Posting journal reviews on preprints – Ludo Waltman (CWTS Leiden), James Fraser (UCSF), Cooper Smout (Free Our Knowledge / IGDORE)
What is the evolutionary bridge between a system based around journal-organized peer review and referred preprints? Is anonymity needed? Who prompts you to review? Who owns your review?
2B: Curation and review in the preprint landscape – Hannah Drury, Paul Shannon, Godwyns Onwuchekwa (eLife/Sciety)
In this interactive session we will be brainstorming and discussing answers to some important questions about curation of preprints and how this relates to review. Is curation a useful addition to the space? What motivates researchers to curate, rather than review?
2C: Tackling information overload: identifying relevant preprints and reviewers – Christine Ferguson, Martin Fenner, Daniel Mietchen
Goal 1: To float the problem of information overload in the scholarly literature and a possible solution for preprints (see our recent blog post); to investigate who in the community thinks this is a problem that needs solving for preprints; who in the community might be working on something similar & who is keen to get involved in a short term-focus group to work on a solution.
Goal 2: Discuss options for finding suitable reviewers for a given preprint.