Organizer
Nokome Bentley, Alexander Ketchakmadze & Colette Doughty from Stencila
Website or social media links
Current stage of development
Ongoing project
Project duration
1 – 3 months for integrations described here as part of Stencila’s ongoing development
Update
Our aim for the #PrePrintSprint was to use it as an opportunity to add reviewing functionality to our platform for executable documents. So, since the kick off we’ve been heads down doing a lot of coding.
We are happy to report that we managed to get most of our tickets moved from the “To do” column to the “Done” column! We’ve added the ability to mint DOIs for both preprints, and reviews as well as integrations to extract reviews from Google Docs and GitHub pull request reviews.
How has your project changed?
We found that there was more work involved in designing and implementing a workflow for requesting reviews and allowing potentially anonymous reviewers to accept or decline those.
Have you integrated any feedback received?
We got some really useful ideas and suggestions from the breakout room at the sprint kickoff. We have only just got our development work completed to a stage where we can start seeking feedback from users. We’d love to get some feedback from people after the sprint.
Have you started any collaborations?
Yes, we have started exploring options for collaboration with PREreview. We are also talking to a journal who is interested in our integration with Github, including the ability to pull reviews from there.
Project aims
Background information on current practices
Preprints are rapidly becoming an integral part of the modern scientific workflow, dramatically increasing the speed at which new research is disseminated. However, although one of the benefits of preprints is their availability for early review, the number of preprints with reviews is relatively low.
Overview of the challenge to overcome
Part of the reason for the scarcity of preprint reviews is the burden it places on both authors and reviewers. Reducing this burden by reducing the friction in the workflow, for both authors and reviewers, provides an opportunity to both increase the number and speed of preprint reviews that are done and acted upon.
The ideal outcome or output of the project
Part of the reason for the scarcity of preprint reviews is the burden it places on both authors and reviewers. Reducing this burden by reducing the friction in the workflow, for both authors and reviewers, provides an opportunity to both increase the number and speed of preprint reviews that are done and acted upon.
Description of the intervention
This project will aim to dramatically reduce the friction in,
- Authors being able to make their preprint available for review
- Reviewers being able to create reviews on a preprint
- Authors being able to incorporate reviewers’ comments and suggestions and publish new versions of their preprint.
To achieve this our approach will be to “go to where the users are” and make use of existing platforms that already provide excellent facilities for reviewing alternative types of content but which are often overlooked for reviewing research articles.
Our envisaged workflow is:
- Authors upload or link their preprint authored in one of several formats (Microsoft Word, Google Docs, Markdown, RMarkdown, Jupyter Notebooks) to a Stencila Hub project and make it available for review on either Google Docs (most suitable for general reviews) or GitHub (most suitable for reviews of research involving a lot of code).
- Reviewers are invited to make reviews on the platform that the author has selected (Google Docs or GitHub) using the built in facilities for this (Pull Requests on Github, Suggestions and Comments on Google Docs).
- Authors respond to comments and suggestions from reviewers using the normal mechanisms for each platform (e.g. accepting changes in GitHub PRs; accepting or rejecting suggested changes on Google Docs).
- Authors create new versions of their preprint (a new “named version” on Google Docs; a “release” on GitHub).
- A new HTML version of the preprint is automatically published on Stencila Hub (using our Webhook integrations with these platforms) with a newly minted DOI and both authors and reviewers are notified.
Plan for monitoring project outcome
We will measure the success of this project by monitoring:
- The number of reviews created (Reviewers making comments on Google Docs, Pull Requests on GitHub).
- The number of revisions of preprints created in response to reviews.
What’s needed for success
Additional technology development
Stencila’s open source software, principally Hub (integration with other platforms) and Encoda (conversion between formats), already have the technical foundations necessary for this project. Some additional development and testing will be required for the API integrations between Stencila Hub and Crossref, Google Docs and GitHub. Refinements to format conversions may also be required.
Feedback, beta testing, collaboration, endorsement
User feedback, in particular one-on-one user testing of the workflows for both authors and reviewers, will be important for the success of this project. We will draw on our existing pool of beta testers for this. Endorsements by ASAPbio, funders and journals will help substantially with uptake.
Funding
The initial integrations with Google Docs and GitHub will be developed as part of Stencila’s ongoing development and do not require any additional funding. If these prove successful funding may be needed for refinements and/or integrations with other platforms.
1 Comment