On December 9, 2019, ASAPbio, EMBO Press, and 17 affiliate journals launched Review Commons, a platform providing independent peer review prior to journal submission. At two subsequent community meetings (one in-person at the ASCB|EMBO conference and the other online the following week) the Review Commons team shared their motivations, planned workflow, and future vision. We also heard questions, concerns, and ideas about the project, which are summarized here as a composite of the two events.
Why Review Commons?
The speed of scientific communication has not
kept pace with tremendous advances in web-based technology. For example, graduates
students are taking on average one year longer to publish their
first author work than they were 30 years ago, and the number of figure panels
per journal article has dramatically increased over the same period. One reason
for these rising expectations may be that the existing journal publication
system performs many different functions for academic research, including
dissemination, evaluation, and curation. Despite more widespread adoption of
preprints to quickly disseminate research findings, evaluation of scientists is
tied to peer review via the perceived prestige of the journal they publish in.
Consequently, there’s pressure to combine many results into a single “big
story” in an effort to publish in a high impact journal, and some papers are
serially resubmitted to many journals in an effort to
find the most prestigious possible home. This inefficient process depletes
valuable peer review time and resources, leading to an estimated 15 million hours of wasted reviewer time per
year.
Enter Review
Commons, “a small village,” in the words of Ron Vale (founder of ASAPbio),
in which collaborators are working to do something good for science. Ron
explained that the goal is to inspire a cultural change in which reviews are
written in a spirit of partnership with the author for the improvement of
science. In this model, reviewers focus on the science rather than journal fit.
Moreover, Review Commons aims to make
the publishing process more efficient by facilitating better match-making
between scientists and journals. ASAPbio and EMBO are approaching Review Commons as an experiment in peer
review.
Ron Vale & Jessica Polka
Initial reactions to the platform
Maria Leptin (Director of EMBO) reminded the
audience that the success of the project depends on engagement and adoption by
the research community. Fortunately, the announcement of Review Commons was met with huge enthusiasm: during formation of
the scientific advisory board, out of 109 invitations sent, 80 were accepted in
less than a week, with only 2 declines. Maria acknowledged that many from the
research community are wondering how Review
Commons will work if the referees don’t know which journal the manuscript
is destined for. This reveals that our concept of peer review is deeply
entangled with journal brands; Review
Commons can help to differentiate between the scientific strength of a
manuscript and the editorial assessment of suitability for publication in a
specific journal. However, Review Commons
is a limited-time trial, and by the end of 2020, the project will have to
develop a sustainable and scalable business model.
Journal-agnostic, high-quality
peer review
Review
Commons aims to decouple the many aspects of
publishing (dissemination, evaluation, and curation) by reversing the canonical
sequence of journal selection and peer review. The final output will be a peer
reviewed preprint that can be submitted, with the platform’s facilitation, to
one of 17 affiliate journals. The scope is currently in the life sciences,
where EMBO Press is confident in obtaining high-quality peer review. Thomas
Lemberger (Deputy Head of Scientific Publications and Review Commons project lead at EMBO) emphasized that reviewers will
be asked to evaluate the submitted work for what it is, not what it should or
can be to fit the editorial standards of a specific journal. To learn more
about the work flow, please watch Thomas’ detailed description in the recording.
Screengrab from Thomas Lemberger’s presentation
EMBO Press and ASAPbio will evaluate Review Commons periodically and use this
information to adapt the workflows and policies.
Journals benefit from peer review
focused on the science
Why engage as an affiliate journal? Bernd
Pulverer (Head of Scientific Publications, EMBO) explained that Review Commons will help affiliate
journals adopt a more transparent publishing process in which reviews are
posted on preprints (see the first Refereed Preprint available on February
3, 2020). They will also benefit from the sequential manuscript transfer
network from the platform to partner journals: since the first set of reviews
is coordinated centrally by Review
Commons, journals do not have to re-assign new referees for re-review of
transferred manuscripts. This dramatically increases the overall efficacy of
the peer review process. With Review
Commons, access to review reports before
submitting to a journal can help authors choose an appropriate venue, which
further streamlines the editorial process. Through these measures, Review Commons can help conserve
editorial resources, potentially reducing the number of reviews per paper from
9-12 by the time of publication to 3-4. Because the platform may attract a new
demographic of authors interested in transparent and innovative publishing,
there may be a net increase in submissions to affiliate journals. The process
may be challenged by: 1) too few or too many submissions, 2) poor quality
submissions leading to difficulty in securing referees, 3) referees that do not
comply with requests to be journal agnostic, or 4) an uneven distribution of Review Commons transfers to certain
partner journals. EMBO and ASAPbio will monitor all of these possibilities.
Q&A
After the presentations in both events,
audience members raised questions and made comments during Q&A periods.
Some of those questions and corresponding answers from EMBO & ASAPbio are
summarized below.
The review process
How does the initial triage by
the Review Commons managing editor
work?
The initial triage decision is made by the
managing editor in consultation with the advisory board. To maintain a high-quality review process
and conserve the finite resource of reviewer time, preselection at the initial
triage stage is kept quick and stringent, sending only studies that represent
significant advances to review.
How long will the peer review
process take? Could this process be faster than the journal-organized peer
review?
A round of review at Review Commons may not be faster than review at affiliate journals
(approximately 28-30 days). If anything, it might be sometimes more challenging
to find reviewers because the platform is new. The biggest time saver is not
the duration of a round of peer review, but rather the reduction of serial
submission and repetition of that process at other journals.
Why is bioRxiv the only preprint
server partner?
This decision was purely practical; bioRxiv is
the dominant preprint server in the life sciences, and it supports the technology that makes it possible to post the
reviews. Other preprint servers may be incorporated in the future.
Could the Review Commons Refereed Preprint be a final endpoint for a paper?
Yes, there is no obligation to submit to an
affiliate journal. These decisions will be tracked as a part of the analysis of
the platform. Authors can also exit the Review
Commons system and submit to another journal with the peer review reports
they received.
Transparency and data sharing
Will cross-commenting be
anonymous? Can reviewers disclose their identities to one another?
Believing that the most important component of
reviews is the scientific content, Review
Commons will conduct cross-commenting anonymously, unless reviewers decide
to sign their reviews.
What level of transparency will
there be about time to transfer, time in review, cost savings, etc?
The time of review will be posted on a
refereed preprint. Other results will be reported in aggregate to report on Review Commons.
Will the editorial decision of
the affiliate journals be visible to each other?
No, the decisions of the affiliate journal
will not be visible to other affiliate journals during transfer. If authors are
unsatisfied with their reviews, they can withdraw from Review CommonsGuidelines for Authors, “Review Commons reserves the right to reject manuscripts at any
point if ethical, biosecurity or scientific integrity issues arise. In serious
cases, Review Commons may post a note
of editorial concern next to the public reviews.”
Future perspectives for Review
Commons
How will Review Commons scale up?
Currently there are 16 EMBO scientific editors
accessing a network of 20,000 registered referees, which would otherwise take
several years to build from scratch. In the future, EMBO may consider deploying
additional editors from affiliate journals, increasing the scope of research
reviewed, and including other journals in the affiliate network.
Would Review Commons consider partnering with community review projects?
Perhaps, but probably not in the beginning due
to limited resources. Efforts are underway to make Review Commons more interoperable and portable for potential
integration with other peer review projects.
Public perception and cultural
change in publishing
Will journals view Review Commons Refereed Preprints as
publication?
Review
Commons will not make an editorial accept/reject
decision, which is one of the features distinguishing it from journals. That
said, it’s unclear whether some journals outside of the affiliate network will
view this as competition in publishing.
Who will consider a Review Commons Refereed Preprints as
valid as journal publication?
It will depend on the different contexts of
‘validity.’ Valid for readers keeping up with the latest science? Valid for
career assessment? Valid for funding? It will be interesting to see how Refereed Preprints are adopted across
these use cases.
Are you ready to send a manuscript to Review Commons? Contribute to the discussion by following #RefereedPreprint and @ReviewCommons.
On December 9, 2019, ASAPbio, EMBO Press, and 17 affiliate journals launched Review Commons, a platform providing independent peer review prior to journal submission. At two subsequent community meetings (one in-person at the ASCB|EMBO conference and the other online the following week) the Review Commons team shared their motivations, planned workflow, and future vision. We also heard questions, concerns, and ideas about the project, which are summarized here as a composite of the two events.
Why Review Commons?
The speed of scientific communication has not
kept pace with tremendous advances in web-based technology. For example, graduates
students are taking on average one year longer to publish their
first author work than they were 30 years ago, and the number of figure panels
per journal article has dramatically increased over the same period. One reason
for these rising expectations may be that the existing journal publication
system performs many different functions for academic research, including
dissemination, evaluation, and curation. Despite more widespread adoption of
preprints to quickly disseminate research findings, evaluation of scientists is
tied to peer review via the perceived prestige of the journal they publish in.
Consequently, there’s pressure to combine many results into a single “big
story” in an effort to publish in a high impact journal, and some papers are
serially resubmitted to many journals in an effort to
find the most prestigious possible home. This inefficient process depletes
valuable peer review time and resources, leading to an estimated 15 million hours of wasted reviewer time per
year.
Enter Review
Commons, “a small village,” in the words of Ron Vale (founder of ASAPbio),
in which collaborators are working to do something good for science. Ron
explained that the goal is to inspire a cultural change in which reviews are
written in a spirit of partnership with the author for the improvement of
science. In this model, reviewers focus on the science rather than journal fit.
Moreover, Review Commons aims to make
the publishing process more efficient by facilitating better match-making
between scientists and journals. ASAPbio and EMBO are approaching Review Commons as an experiment in peer
review.
Initial reactions to the platform
Maria Leptin (Director of EMBO) reminded the
audience that the success of the project depends on engagement and adoption by
the research community. Fortunately, the announcement of Review Commons was met with huge enthusiasm: during formation of
the scientific advisory board, out of 109 invitations sent, 80 were accepted in
less than a week, with only 2 declines. Maria acknowledged that many from the
research community are wondering how Review
Commons will work if the referees don’t know which journal the manuscript
is destined for. This reveals that our concept of peer review is deeply
entangled with journal brands; Review
Commons can help to differentiate between the scientific strength of a
manuscript and the editorial assessment of suitability for publication in a
specific journal. However, Review Commons
is a limited-time trial, and by the end of 2020, the project will have to
develop a sustainable and scalable business model.
Journal-agnostic, high-quality
peer review
Review
Commons aims to decouple the many aspects of
publishing (dissemination, evaluation, and curation) by reversing the canonical
sequence of journal selection and peer review. The final output will be a peer
reviewed preprint that can be submitted, with the platform’s facilitation, to
one of 17 affiliate journals. The scope is currently in the life sciences,
where EMBO Press is confident in obtaining high-quality peer review. Thomas
Lemberger (Deputy Head of Scientific Publications and Review Commons project lead at EMBO) emphasized that reviewers will
be asked to evaluate the submitted work for what it is, not what it should or
can be to fit the editorial standards of a specific journal. To learn more
about the work flow, please watch Thomas’ detailed description in the recording.
EMBO Press and ASAPbio will evaluate Review Commons periodically and use this
information to adapt the workflows and policies.
Journals benefit from peer review
focused on the science
Why engage as an affiliate journal? Bernd
Pulverer (Head of Scientific Publications, EMBO) explained that Review Commons will help affiliate
journals adopt a more transparent publishing process in which reviews are
posted on preprints (see the first Refereed Preprint available on February
3, 2020). They will also benefit from the sequential manuscript transfer
network from the platform to partner journals: since the first set of reviews
is coordinated centrally by Review
Commons, journals do not have to re-assign new referees for re-review of
transferred manuscripts. This dramatically increases the overall efficacy of
the peer review process. With Review
Commons, access to review reports before
submitting to a journal can help authors choose an appropriate venue, which
further streamlines the editorial process. Through these measures, Review Commons can help conserve
editorial resources, potentially reducing the number of reviews per paper from
9-12 by the time of publication to 3-4. Because the platform may attract a new
demographic of authors interested in transparent and innovative publishing,
there may be a net increase in submissions to affiliate journals. The process
may be challenged by: 1) too few or too many submissions, 2) poor quality
submissions leading to difficulty in securing referees, 3) referees that do not
comply with requests to be journal agnostic, or 4) an uneven distribution of Review Commons transfers to certain
partner journals. EMBO and ASAPbio will monitor all of these possibilities.
Q&A
After the presentations in both events,
audience members raised questions and made comments during Q&A periods.
Some of those questions and corresponding answers from EMBO & ASAPbio are
summarized below.
The review process
How does the initial triage by
the Review Commons managing editor
work?
The initial triage decision is made by the
managing editor in consultation with the advisory board. To maintain a high-quality review process
and conserve the finite resource of reviewer time, preselection at the initial
triage stage is kept quick and stringent, sending only studies that represent
significant advances to review.
How long will the peer review
process take? Could this process be faster than the journal-organized peer
review?
A round of review at Review Commons may not be faster than review at affiliate journals
(approximately 28-30 days). If anything, it might be sometimes more challenging
to find reviewers because the platform is new. The biggest time saver is not
the duration of a round of peer review, but rather the reduction of serial
submission and repetition of that process at other journals.
Why is bioRxiv the only preprint
server partner?
This decision was purely practical; bioRxiv is
the dominant preprint server in the life sciences, and it supports the technology that makes it possible to post the
reviews. Other preprint servers may be incorporated in the future.
Could the Review Commons Refereed Preprint be a final endpoint for a paper?
Yes, there is no obligation to submit to an
affiliate journal. These decisions will be tracked as a part of the analysis of
the platform. Authors can also exit the Review
Commons system and submit to another journal with the peer review reports
they received.
Transparency and data sharing
Will cross-commenting be
anonymous? Can reviewers disclose their identities to one another?
Believing that the most important component of
reviews is the scientific content, Review
Commons will conduct cross-commenting anonymously, unless reviewers decide
to sign their reviews.
What level of transparency will
there be about time to transfer, time in review, cost savings, etc?
The time of review will be posted on a
refereed preprint. Other results will be reported in aggregate to report on Review Commons.
Will the editorial decision of
the affiliate journals be visible to each other?
No, the decisions of the affiliate journal
will not be visible to other affiliate journals during transfer. If authors are
unsatisfied with their reviews, they can withdraw from Review CommonsGuidelines for Authors, “Review Commons reserves the right to reject manuscripts at any
point if ethical, biosecurity or scientific integrity issues arise. In serious
cases, Review Commons may post a note
of editorial concern next to the public reviews.”
Future perspectives for Review
Commons
How will Review Commons scale up?
Currently there are 16 EMBO scientific editors
accessing a network of 20,000 registered referees, which would otherwise take
several years to build from scratch. In the future, EMBO may consider deploying
additional editors from affiliate journals, increasing the scope of research
reviewed, and including other journals in the affiliate network.
Would Review Commons consider partnering with community review projects?
Perhaps, but probably not in the beginning due
to limited resources. Efforts are underway to make Review Commons more interoperable and portable for potential
integration with other peer review projects.
Public perception and cultural
change in publishing
Will journals view Review Commons Refereed Preprints as
publication?
Review
Commons will not make an editorial accept/reject
decision, which is one of the features distinguishing it from journals. That
said, it’s unclear whether some journals outside of the affiliate network will
view this as competition in publishing.
Who will consider a Review Commons Refereed Preprints as
valid as journal publication?
It will depend on the different contexts of
‘validity.’ Valid for readers keeping up with the latest science? Valid for
career assessment? Valid for funding? It will be interesting to see how Refereed Preprints are adopted across
these use cases.
Are you ready to send a manuscript to Review Commons? Contribute to the discussion by following #RefereedPreprint and @ReviewCommons.
By Victoria Yan