2022-01-13 update: The FAST principles have now been posted as a preprint.
As Chris Jackson pointed out in the first session of #FeedbackASAP, preprint feedback is part of the collaborative culture of science. A positive culture of comments and review of preprints brings many benefits to the scientific process, but what does this culture look like in practice?
We dedicated one breakout session during the #FeedbackASAP meeting to explore questions around preprint feedback culture. During the session we focused on two main aspects: a discussion of the FAST principles for preprint feedback, and an exploration of what needs to happen so that we get to the positive culture we wish to see.
The FAST principles
We first covered an overview of the FAST (Focused, Appropriate, Specific, Transparent) principles for preprint feedback developed by the ASAPbio preprint review cultural norms Working Group, and we encouraged the group of attendees to provide feedback on how the principles could be clarified or improved.
Focused
- Respect the focus of the paper
- Focus on the science and not the journal
- Focus on the science and not the people
Appropriate
- The tone should always be polite, respectful and constructive
- Consider your motivation for providing feedback – if not constructive, reconsider
- Reflect on any biases before, during and after commenting, including your background, professional role, perception about the work’s interest level
- Public feedback is a form of scientific discourse and should be approached as such by all parties
- There is a collective responsibility to call out inappropriate behaviors or critiques
- Behave with integrity, following ethical conduct expected in all research activities
Specific
- Feedback should be specific, clear, actionable, and useful for the authors to improve the work
- Distinguish critical vs optional items
- Are claims supported by the data
- Candid
Transparent
- Share information on limits of expertise and competing interests
- Post feedback publicly, if comfortable doing so, sign the comments
- Acknowledge errors
- Disclose & credit colleagues who contributed
The attendees discussed the public nature of preprint feedback. In the context of preprint feedback, the commenter is placing the comments publicly and it could be argued that the primary audience of the feedback is the reader, and the commentary does not need to necessarily cater for the author. This brings a potential tension for the reviewer between being candid about any flaws in the work and keeping a respectful tone toward the authors; with this in mind, it was suggested that commenters should strive to be ‘honest but merciful.’
The attendees also commented on the role of different actors in the preprint feedback exchange and suggested clearly designating for whom each of the principles is intended. There were also questions about the role of journals in this context: How can journals participate in encouraging use of the FAST principles? If a journal does not operate open peer review, how should it approach feedback posted publicly on the preprint record?
In terms of possible gaps in the FAST principles, attendees suggested defining expectations for the authors’ response to any feedback on the preprint. Are authors expected to respond to feedback, and if so, what would be the appropriate avenue? Should there be some form of rebuttal/appeals process?
Another salient point in the discussion related to the role of moderation. While the principles outline behaviors to engage in feedback in a polite and constructive manner, there are examples in other forums (e.g. comments on newspaper or magazine articles) where public commenting can go off topic or diverge from collegial exchanges. There should be some consideration as to how exchanges would be moderated, what steps should be taken if preprint feedback is not completed per the FAST principles, and whose responsibility it would be to take any such steps.
Creating a positive preprint feedback culture
In the second part of the session, we asked participants to envision a future where a positive culture of preprint feedback is the norm and to reflect on how the FAST principles helped us get there, and how they are being used in that context.
The session participants shared ideas on what preprint review would look like in that ecosystem. There would be many more preprint reviews, contributed by different stakeholders including overlay journals. In that future scenario, participation in preprint review is supported by student training that incorporates the FAST principles as part of the courses (in training run by institutions as well as journal clubs). Another important element that attendees highlighted was the recognition of preprint reviews as part of job and grant applications, that is, preprint review would be recognized as a scholarly activity in its own right. Attendees mentioned that the FAST principles could become a standard by which preprint reviews would be expected to abide to earn recognition, providing a badge of sorts for the quality and appropriateness of the preprint review.
We also asked session attendees to reflect on how that culture of preprint review impacted early-career researchers (ECRs). In addition to highlighting the importance of receiving credit for preprint reviews (e.g. consideration as part of job interviews), there were also comments referring to ECRs showing enthusiasm and confidence in both providing and receiving reviews, and broader recognition of the ECRs’ expertise and valid role as reviewers. However, during the discussion, attendees noted that there were specific considerations on the impact of preprint feedback for ECRs that will need their own dedicated consideration as preprint commenting culture evolves.
In a broader reflection of how we can drive progress toward the preprint feedback culture we’d like to see, attendees reflected on the fact that we are at a relatively early stage of preprint feedback activities. At this stage, any principles should not be overly prescriptive, but rather provide broad guidelines that allow experimentation in this space. We should thus expect principles to evolve as participation in preprint review develops in the future.
The session provided valuable feedback for the preprint review cultural norms Working Group to iterate on the FAST principles and take the discussion forward about their future implementation. We thank all attendees for their contributions and we invite the community to continue to share feedback by commenting on this blog post or directly on the FAST principles document.
