By Vivian Siegel
In my own experience, and I’ve written about this in the past, peer review in the context of journal submission suffers from a number of biases. These include journal-based biases that would be eliminated by a journal agnostic process. I summarize the main points below.
We all know reviewers are biased (why? Because we’re human).
We’re biased based on our own history – how our own experiments have led to our view of the way a particular phenomenon works – and are much more skeptical of experiments that go against our views than we are of those that align with them. To the editors who expect a controversial paper to be enthusiastically endorsed by reviewers on both sides of the controversy: good luck.
We are also biased based on the success or failure of our own journal submissions, and on our preconceptions of “what the journal wants”. When I was an editor at Cell, I quickly realized that many reviewers would be more critical of papers in their field if they had recently had a paper rejected from Cell than if they had a paper accepted.
But even beyond that, we are biased by what we think the journal wants to publish. Ben Lewin told me that when he decided to include structural biology in Cell, it took him years before reviewers would review the paper without providing the knee-jerk “this is not a Cell paper; you don’t publish structures” response. I experienced it when we started publishing short papers (“Cell doesn’t publish short papers”), and we’ve all seen reviews where the reviewer simply says “This isn’t a [INSERT JOURNAL NAME OF YOUR CHOICE] paper – it’s not mechanistic enough or trendy enough, or long enough, or [SOMETHING] enough.” It’s easy to dismiss a paper when we think we know what the journal wants.
In contrast, I found time and again that when a journal is new (and I’ve helped launch several journals, including Molecular Cell, Developmental Cell, PLOS Biology, and Disease Models & Mechanisms), peer reviews would take on a different quality. Reviewers would write something like, “I don’t know how to advise you because I don’t know what your standards are or what you want in the journal, so I guess I’ll just tell you the strengths and weaknesses of the paper and you will have to decide.” (Thank you, this is all I ever wanted). At the same time, authors would remark that the reviews were unusually constructive – whether we chose to accept or reject the paper. These are anecdotal observations, and I would love to collect some data on the quality of review when you don’t know journal identity.
Back in my days as a journal editor, not many biologists were posting preprints, so my idea was to create a “journal blind” submission system during standard peer review. But with BioRxiv and other preprint servers, performing peer review in advance of journal publication becomes possible. I encourage the community to try it, gather some data, and proceed based on evidence.
