Public feedback on preprints can unlock their full potential to accelerate science.

Public preprint review can help authors improve their paper, find new collaborators, and gain visibility. It also helps readers find interesting and relevant papers and contextualize them with the reactions of experts in the field. Never has this been more apparent than in COVID-19, where rapid communication and expert commentary have both been in high demand. Yet, most feedback on preprints is currently exchanged privately.

Join ASAPbio in partnership with DORA, HHMI, and the Chan Zuckerberg Initiative to discuss how to create a culture of constructive public review and feedback on preprints.

Meeting registration

July 21, 2021 | 15:00 UTC: 8am PDT, 11am EDT, 4pm UK, 5pm CEST, 8:30pm IST | See more time zones | Meeting duration: 4 hours

We invite researchers and others interested in preprint review to join us to share your ideas, shape the conversation, and start or join new projects. Register by June 30 for the opportunity to submit a brief (1-2 sentence) proposal for your own breakout session. 

The meeting is free and we welcome participation from everyone, but registration is required. After you register below, you’ll receive an email confirmation that will allow you to add the event to your calendar.

 

 

Agenda (in progress)

Time in UTC (duration)Session description
15:00Welcome & logistics
15:05Why preprint feedback?
15:35Institutional incentives for preprint feedback
16:00What can you do? 
16:25

Breakout session 1:

  • Cultural norms: How can authors, readers, reviewers, and evaluators interpret and react to preprint feedback? 
  • Taxonomy: Amid a variety of different approaches to providing feedback on preprints, what are the most important attributes we need to consider and convey to users, and a shared language for doing so?
17:45Breakout session 2: Attendee-led sessions
18:55Wrapup and conclusion
19:00Adjourn

Take action to support preprint review

As an author

Why request public feedback on your preprint?

By explicitly requesting public feedback on your preprint, you’ll help readers feel welcome to share their comments. By specifying the time frame and type of feedback that would be most useful to you, the input you get is more likely to be constructive. And, by tagging your comment or tweet with #FeedbackASAP, you can help to demonstrate momentum for preprint review.

How to do it

Leave a comment on your own preprint inviting public review. Here’s a suggested format:

“My co-authors and I welcome public feedback on our preprint, ideally by [DATE]. We are especially interested in [statistics, etc]. #FeedbackASAP”

Including the hashtag will help us to find your request and demonstrate author support for open commenting.

Author requests for preprint feedback

We monitor the bioRxiv comments and tweets for the #FeedbackASAP hashtag to produce the list below. If you leave a comment elsewhere (such as the comment section of another preprint server), email jessica.polka@asapbio.org to let us know!

 

Other ways to request feedback

On PREreview, any researcher (including authors) can request feedback on any preprint with a DOI as well as provide feedback to preprints in the form of rapid and full PREreviews.

Starting in July 2021, eLife will exclusively review preprints and post public reviews on them.

You can use our registry, ReimagineReview, to identify projects that review preprints. The following listings represent those that allow authors to submit their own preprints for feedback.

As a reader / reviewer

James Fraser (UCSF) has adopted a policy in his lab compact to post all reviews as comments on bioRxiv (emphasis ours):

 

“We believe that transparency in peer review is critical to improving the quality and professionalism of reviews. We only review manuscripts that have been posted on BioRxiv and we post our peer review comments publicly on Biorxiv as well. Peer review is also viewed as a training opportunity and all lab members who co-review with JF are credited for their contribution to the review. JF recognizes that he holds certain privileges that allow him to post peer review comments non-anonymously without fear of repercussion. Unfortunately, this is not the case for all researchers. JF is willing to post reviews on behalf of anyone, both in and outside of the Fraser lab, who wishes to remain anonymous. Guidelines for how to engage in respectful and constructive peer review can be found on the Peer Review in the Life Sciences course page.”

 

You can use our registry, ReimagineReview, to identify projects that review preprints. The following listings represent those that allow authors to submit their own preprints for feedback.

Preprints have great potential to democratize access and production of knowledge. However, researchers tend to be most aware of work occurring within their own network; highlighting and sharing only these preprints can contribute to the Matthew effects and limit readers’ exposure to a sliver of available science. Looking outside of the obvious candidates when highlighting or amplifying preprints can help. You can:

  • Search regional servers. Because some servers may overrepresent certain countries (Abdill et al., 2020), check regional preprint servers such as AfricArxiv and RINarxiv to find papers that may be less visible to your colleagues outside those regions, or include the names of specific countries in your search to find work from them. 
  • Publish and search preprints in various languages. An increasing number of preprint repositories accept submissions in more than one language. Make strategic searches by keywords in English and their translations to your mother tongue. Check if a preprint already has one or more translations (of the abstract) available. For more on the importance of multilingualism, see the Helsinki initiative. 
  • Find untweeted preprints. For example, when looking through bioRxiv for her #365preprints project (see below), Prachee Avasthi recommends selecting preprints that have yet to be tweeted. 

Thanks to Dasapta Erwin Irawan, Jo Havemann, and Stefano Vianello for their input.

In addition to using established tools, you can also create your own preprint highlights in a variety of formats.

For example, Prachee Avasthi uses Twitter Spaces to host a short audio conversation about an interesting preprint every day.

Jonny Coates, Emma Wilson, and John Howard are starting a podcast for longer discussions:

(Note that we’re piloting a program to provide support for mission-oriented expenses as part of our community program)

As a journal

eLife has announced that from July 2021, it will exclusively review preprints and post public reviews on them. According to the press release,

“eLife is refocusing its editorial processes towards transforming preprints into ‘refereed preprints’ that include a public assessment of the work prepared by the journal’s reviewers and editors. This will involve providing updated instructions for its reviewers to capture what should and should not be included in public reviews, and modifying its editorial processes to focus on the production of these reviews. “Our plan is to work closely with our editors, reviewers, authors and readers to optimise the process by which public reviews are created and their utility to the diverse audiences we hope to reach,” Eisen says.”

Related projects

Norms working group

The Working Group will produce 1) a set of recommended cultural norms for participation in preprint review, 2) a library of available resources relevant to constructive review of preprints, and 3) materials to support dissemination of the above. The working group is chaired by Iratxe Puebla.

  • Maryrose Franko (Health Research Alliance)
  • Sharon Ahmad (The Company of Biologists)
  • Hilda Bastian (hildabastian.net)
  • Sara Monaco (Review Commons / EMBO)
  • Dyche Mullins (UCSF)
  • Samantha Hindle (PREreview)
  • Sandra Franco Iborra (New York Genome Center)
  • Gautam Dey (EMBL)
  • Shriyaa Mittal (Harvard Medical School/MGH)
  • Tim Behrens (University of Oxford)

Taxonomy working group

In order to facilitate the display, filtering, and interpretation of varied forms of preprint feedback, the taxonomy working group will create a simple, shared vocabulary for describing the most valued characteristics of preprint review. The working group is chaired by Victoria Yan.

  • Gabe Stein (KFG/Doc Maps)
  • Gary McDowell (KFG/Doc Maps)
  • Tony Ross-Hellauer (TU Graz/Doc Maps)
  • Kathleen Shearer (COAR Notify)
  • Philip Cohen (SocArxiv)
  • Thomas Lemberger (EEB)
  • Phil Hurst (Royal Society)
  • Clare Stone (SSRN)
  • Damian Pattinson (eLife)
  • Richard Sever (bioRxiv, medRxiv)