By Iain Cheeseman
Whitehead Institute

Journals play a critical role in the scientific process, refining research through peer review and disseminating it to appropriate communities. At its best, the publishing process is a partnership among editors, staff, authors, reviewers, and readers. Each group has a vested interest in working together to ensure a robust and fair editorial evaluation, rigorous and constructive review, and the broadest visibility of the work. This process functions best when each group communicates openly with the others as we strive to refine the way that science is conducted and disseminated.

The last few years have seen a revolution in life sciences publishing with the adoption of preprints and increasingly diverse experiments from traditional journals. A recent meeting organized by ASAPbio, HHMI, and Wellcome focused on bringing more transparency and innovation to our peer review system. One of the most important tangible recommendations to come from the meeting, in my opinion, was the proposal that journals should widely adopt more transparency in the peer review process. There was near unanimous consensus from the participants—which included scientists, representatives of journals, scientific societies, and funders—that more journals should commit to publishing the peer reviews of every paper that appears in the journal. Similar to the peer review processes offered by EMBO Journal, eLife, and others, the reviewers could still remain anonymous, but this transparency would provide much better insight into the assessment process and nature of the paper.

Make your voice heard: writing letters to the editor

As an author, reviewer, and editorial board member, I fully support publishing the contents of peer reviews. But how do we as individual members of the scientific community work to effect this change? We must be vocal. The more that journals hear about the desire for a transparent process, the more likely it is to be quickly adopted—similar to the wildfire-like spread of preprints.

If you are an author or reviewer, you can request transparent peer review when you have a paper accepted or you submit your review. If you are a member of editorial board, advocate for this to your journal team. When you meet an editor at a scientific meeting or during a lab visit, give them your perspective. Or simply write a letter out of the blue. Editors are remarkably engaging and receptive to things that can make science better.

To help you in your own letter writing campaign, I have included my arguments in favor of adopting a transparent peer review process. Feel free to take and adapt these ideas for your own advocacy. And add your own suggestions and arguments as comments or annotations using the Hypothesis toolbar at right. This is modeled on letters that I have been sending myself, and I urge you to be the instrument of change to make science better.

Perspectives on the benefits of open peer review reports

I would particularly like to highlight the following arguments in favor of transparent peer review:

1. As a member of editorial boards, I think that journals should be proud of the nature of their review process. The hard work from editors adds substantial value to the publication process. From speaking with diverse editors, it is clear how much thought and deliberation goes into every paper, even if I don’t agree with every decision. Unfortunately, I don’t think that journals get enough credit for this. Publishing the reviews (and editorial assessments) would highlight how fantastic and deliberative this process typically is.
2. As an author, I believe that reviewers knowing that their comments will ultimately be made public will encourage them to write higher quality and more constructive (but still critical) reviews. I value the feedback that we obtain through the review process, and I feel that this change will help improve this even further.
3. As a reader, having access to reviews enhances my understanding of a paper by bringing to light concerns and perspectives that I might not have thought of. This is true of the reviews themselves, and also of the authors’ responses; the latter often provides the opportunity to highlight a point that can’t be contained within the paper itself. The inability to see these thoughts, points, and responses robs us of important information for considering the published work.
4. As a mentor, it is complicated that the first reviews that a student (or post-doc) often sees are critiques of their own first paper. Examples and models are critical for training others how to constructively contribute, both as authors and reviewers.
5. As an educator, the inclusion of reviews is remarkably useful in teaching the literature. For the paper-reading courses that I have taught, I have had to contact the authors directly to get their reviews. However, when I have done this, the students have LOVED it. They have really learned a lot from reading these, both about why certain aspects of a paper are included and the peer review process in general.
6. In an era where fairness, reproducibility, and other issues are highly visible in the scientific process, I strongly believe that transparency can help shine a light on things and make a substantial difference in how we evaluate and consider science. Let’s lift the curtain so that everyone can see and understand the process along with all of its benefits and occasional limitations.

Responding to concerns

I firmly believe that publishing peer reviews is the right thing for all of the stakeholders in science. However, in conversations about publishing peer reviews, some concerns generally emerge.

It’s too costly.

Many journals operate on a tight budget. Although the process of consolidating the reviews and preparing them for publication as a PDF isn’t extensive, collectively it can add substantial time (and therefore cost) to the process of preparing a manuscript. Eventually, improved automation within content management systems could alleviate this concern. However, in the meantime, authors can make it clear that you would support a nominal increase in the page charges to support the 15-30 minutes of editorial time necessary to facilitate this. It would be even better if funding agencies could recognize this cost, too.

How do you handle papers that are transferred between journals?

As either an author or reviewer, I would be comfortable to have reviews from a different journal that were transferred together with the paper included as part of a transparent review file online. This is something that could be addressed by providing consent when one completes the review process, much in the same way that many journals ask for permission to transfer the reviews for consideration to another journal. However, when reading the reviews for a transferred paper, it is important to highlight that people will review a paper differently depending on what journal it is for. Clearly indicating that the review represents comments for a different submission is important for evaluating the review in context.

 What if published reviews are taken out of context?

When we write reviews, we typically imagine a small audience of the authors and the editor. Knowing that your review may be read by thousands of people creates distinct challenges. This includes those who may wish to use a negative review to target a paper or an author unfairly. It is important to continue to educate the broader community and lay public on both the strengths and challenges of the peer review process. It is also important to note that some reviewer comments would not be relevant to the final published version of a paper (for example, if the authors take out a line of experimentation, or if, in the case of review transfer, the reviewer provides comments specifically directed at relevance or impact for the original journal). Ultimately, a review represents one person’s opinion at a single moment in time, and it is critical to help people interpret what this means. Importantly, the more reviews that are made publicly available, the easier it will be for people to broadly see and understand what these mean and to read them with appropriate context.

Conclusions and Outlook

We currently have a unique opportunity to improve the submission, review, and publishing processes to help make science more robust and transparent. Publishing peer reviews represents an important step for this, and I urge you to advocate for this change whenever you can.

Creating consistency across the review process will also facilitate the clear interpretation of published peer reviews, transfer between journals, and the ability to provide appropriate context for the different elements of a review. I favor a review process that clearly separates out the consideration of impact from the evaluation of technical merits and provides clear and consistent advice to reviewers on the information that they need to provide. For example, there could be several separate text entry boxes on the online form to provide: 1. An overall summary of a paper, 2. Comments regarding the technical considerations that are relevant no matter where a paper is ultimately published, and 3. Comments regarding impact, including the type of experiments that would be needed to prove the claims at the highest level or make it truly impactful. This separation could make the process easier for authors, editors, and reviewers as a paper migrates to a different journal. This would also help provide transparency for the way that a paper was evaluated by ensuring that the technical aspects of a paper and its impact are considered separately.

The review process in its current form has existed for several decades. In many cases, it works well to improve science and provide a robust evaluation process, but when opportunities present themselves to make things even better, we should do whatever we can to implement positive and constructive changes.

---

Iain Cheeseman is a Member of the Whitehead Institute and Associate Professor of Biology at MIT. His lab is focused on understanding the molecular mechanisms that underlie cell division and chromosome segregation, and he is also invested in undergraduate, graduate, and post-doc training and education.

 

Featured image:  Slipping through the cracks by jah~ is licensed under CC BY-NC 2.0