The preprint is a working group product borne out of the, ‘Supporting interoperability of preprint peer review metadata,’ workshop co-organized by Europe PMC and ASAPbio in October 2023. Presenters Maria Levchenko (Europe PMC, EMBL-EBI) and Bianca Kramer (Sesame Open Science) are co-authors of that preprint.
Maria presented first, defining preprint review metadata as “all kinds of information about the review record,” e.g., reviewer name, affiliation, link to preprint being reviewed, review outcome, assigned score, and more. She stressed that proper metadata is essential to making information readable to machines. Properly organized and accessible metadata makes it easy to display information automatically in a user-friendly way. It can also facilitate the discovery of reviews and allow the community to create new tools.
Maria defined the general workflow and all technical definitions needed to understand the sharing of preprint review metadata. Then, she discussed six approaches to preprint review metadata transfer and their pros and cons.
Workflow 1: Preprint server with the integrated review option, e.g., preprints posted on ScienceOpen.
Workflow 2: Preprint review group using a commenting platform, e.g., bioRxiv preprints reviewed by Arcadia Science using Hypothesis.io.
Workflow 3: Preprint review group registering DOIs, e.g., preprint review by Peer Community In (PCI).
Workflow 4: Preprint review group using a repository, e.g., SciELO preprints review by PREreview and shared using Zenodo.
Workflow 5: Open research platforms, e.g., F1000Research, which makes manuscripts publicly available after editorial checks, followed by an open peer review. If the manuscript reaches three positive reviews from experts invited by the platform, it is sent to indexers, such as PubMed.
Workflow 6: Publish-Review-Curate platform. In the first step, the manuscript is made public (Publish step), then it is Reviewed. Finally, in the Curate stage, it is compiled into a collection, or a summary is added, e.g., medRxiv preprints, reviewed and published by eLife.
In the end, Maria discussed the group’s recommendations for a preprint review group seeking to initiate a new service. Maria recommends using one of the six existing systems instead of inventing another one. For a new group to decide which workflow is best for them, they should consider four key factors:
Technical expertise – Do you have technical expertise in a group? Should you use available solutions, or do you need to develop one?
Cost – Do you have funds to cover the cost of registering DOIs?
Maintenance and sustainability – Can you maintain the records your group will create, or should you outsource that?
Discoverability and linking between preprints and reviews – How important is it that your reviews will be discoverable by multiple services and linked to the reviewed preprint?
In the second part of the call, Bianca discussed linking preprints and preprint review metadata. The extent of linking preprints and preprint reviews was analyzed considering platforms that use Crossref and DataCite to deposit preprints and/or metadata. The group limited the analysis to the year 2023.
Bianca described the differences in Crossref and DataCite metadata regarding how they indicate preprints and reviews, explaining the difficulties one needs to consider when extracting specific information and the limitations of the approach they applied.
Ultimately, she provided some recommendations and a summary of their findings:
Technical implementations—Some technical implementation is required to discover the links between preprints and reviews. If the links to preprints or preprint reviews are not included in the metadata, you cannot retrieve them using metadata.
Types of identifiers—Whether metadata allows links to multiple identifier types. For example, Crossref only allows links to DOIs, which limits the discoverability of preprint reviews without DOIs.
Versioning—Are preprint reviews linked to all preprint versions or a specific version? It would be easier if each preprint version had a separate DOI to link each review to one particular version of a preprint.
Completeness of metadata records—Currently, not all metadata fields are filled. For example, reviews on Zenodo do not link to the specific platforms that posted them, eg, if the PREreview platform posted a preprint review, the metadata will not have the information that PREreview was the poster.
Discoverability and linking between preprints and reviews – If we think linking preprint and their reviews is important, then we, as a community, should discuss how to increase the adoption of better practices regarding metadata and the adoption of DOIs or move beyond DOIs.
The talks were followed by a Q&A session, during which participants discussed a few topics. The discussion started with the usefulness of metadata in identifying data trust signals. Maria then discussed Europe PMC’s plans in this direction. Then, the participants talked about the complexity of the PRC model and its evolution.
If you missed the call, you can view it on YouTube!
A recent ASAPbio Community Call (February 26, 2025) was inspired by a preprint titled ‘Mapping the preprint review metadata transfer workflows.’
The preprint is a working group product borne out of the, ‘Supporting interoperability of preprint peer review metadata,’ workshop co-organized by Europe PMC and ASAPbio in October 2023. Presenters Maria Levchenko (Europe PMC, EMBL-EBI) and Bianca Kramer (Sesame Open Science) are co-authors of that preprint.
Maria presented first, defining preprint review metadata as “all kinds of information about the review record,” e.g., reviewer name, affiliation, link to preprint being reviewed, review outcome, assigned score, and more. She stressed that proper metadata is essential to making information readable to machines. Properly organized and accessible metadata makes it easy to display information automatically in a user-friendly way. It can also facilitate the discovery of reviews and allow the community to create new tools.
Maria defined the general workflow and all technical definitions needed to understand the sharing of preprint review metadata. Then, she discussed six approaches to preprint review metadata transfer and their pros and cons.
In the end, Maria discussed the group’s recommendations for a preprint review group seeking to initiate a new service. Maria recommends using one of the six existing systems instead of inventing another one. For a new group to decide which workflow is best for them, they should consider four key factors:
In the second part of the call, Bianca discussed linking preprints and preprint review metadata. The extent of linking preprints and preprint reviews was analyzed considering platforms that use Crossref and DataCite to deposit preprints and/or metadata. The group limited the analysis to the year 2023.
Bianca described the differences in Crossref and DataCite metadata regarding how they indicate preprints and reviews, explaining the difficulties one needs to consider when extracting specific information and the limitations of the approach they applied.
Ultimately, she provided some recommendations and a summary of their findings:
The talks were followed by a Q&A session, during which participants discussed a few topics. The discussion started with the usefulness of metadata in identifying data trust signals. Maria then discussed Europe PMC’s plans in this direction. Then, the participants talked about the complexity of the PRC model and its evolution.
If you missed the call, you can view it on YouTube!