Note: the RFI is now closed. The NIH has announced a policy that encourages the use of preprints.

The NIH has recently released a request for information (RFI) on the use of preprints and other interim research products. We encourage all interested parties to respond to the RFI using the submission website by the deadline of December 9th 2016 (extended from November 29th).

ASAPbio’s draft response is posted below. Even if you completely agree with our draft, we encourage you to submit your own responses as well. A large number of responses will be critical in conveying a strong message of community interest in preprints and other interim research products to the NIH. Responses from individual scientists at all career stages are encouraged. You do not have to respond to all questions, and the responses can be short. If you would like to share comments or your own response to the RFI, please use the comment section below the post.

Other public responses

• David Mobley, UCI Pharmaceutical Sciences
• Dave Bridges, University of Michigan School of Public Health
• Jordan Anaya, creator of PrePubMed and OncoLnc
• Members of ASBMB’s Public Affairs Advisory Committee (PAAC)
• Sandeep Chakraborty, Celia Engineers
• Samantha Hindle, UCSF
• Stephen Floor, UC Berkeley
• preprints.org
• Rescuing Biomedical Research (RBR)
• Prachee Avasthi (University of Kansas Medical Center)
• John Chodera, Memorial Sloan Kettering Cancer Center
• Gary McDowell, Future of Research (personal response)
• eLife
• FASEB (Federation of American Societies for Experimental Biology)
• AAMC (Association of American Medical Colleges)

ASAPbio’s response

1. Types of interim research products your or your organization create/and or host.

ASAPbio (asapbio.org) is a researcher-driven project to promote the productive use of preprints in biology. We have convened meetings to bring together stakeholders (scientists, funders, journals, and technical experts) to discuss the role that preprints can play in life sciences communication. We are also working to increase awareness of preprints among scientists and to promote policies that will enable preprints to function as a form of scholarly communication. This response represents the opinions of our leadership, but individual members will also submit responses.

We consider preprints to be complete manuscripts (data, methods, and interpretation) posted on an established, public server typically prior to or during journal-organized peer review.

Individual scientists on the ASAPbio committee, both junior and senior, are posting preprints to disseminate our scientific work prior to journal publication to receive additional feedback and visibility and to accelerate the general pace of scientific communication.

2. Feedback on what are considered to be interim research products, and how they are used in your field.

We consider preprints to be manuscripts containing complete disclosures of scientific information: data, interpretation, and methods. While they are typically identical to the version of a manuscript submitted to a journal (for either the initial or revised submissions), preprints could also be used to communicate a broader category of works—for example, negative results, single figures, methods, mathematical/computation models, and data sets. Preprints should always include a description of all methods needed to replicate the work.

While preprints are not yet widely used in biology, their adoption is accelerating. We feel that a wider use of preprints would significantly benefit the scientific process.

3. Insight on how particular types of interim research products might impact the advancement of science.

A strong NIH policy concerning preprints will have far-reaching consequences. The development of NIH-endorsed standards for preprints will be critical for their acceptance by the biomedical research community; increased adoption of preprints will benefit both scientists and the research process.

Preprints can accelerate scientific progress in several ways:

A) By quickly communicating new findings to other labs that can build upon the work. In contrast to traditional peer review prior to publication in journals, a process that can sometimes delay the release of findings by several months or even years, the posting of preprints makes the work openly accessible in 1-2 days.

B) By encouraging diverse and more plentiful feedback from peers early in the process of manuscript revision. In contrast to closed journal peer review processes in which 2-3 scientists review a manuscript, the entire scientific community is free to provide feedback and input to authors during the time that a preprint is being revised. As usage of preprints increases, their growing visibility will create the potential for even more commentary.

C) By raising the visibility of research work and promoting collaborations and invitations to meetings. Preprints can connect scientists with similar interests to one another earlier in the research process, presenting more opportunities to work together.  

D) By enabling scientists to demonstrate recent progress in the pursuit of research goals, productivity as demonstrated by preparation of a manuscript, and willingness to share findings with colleagues and the public before formal publication. These features can help authors with career advancement, with better informed review of grant applications, and with the establishment of a record of independent work. This is especially important for early career researchers, who are likely to have a limited publication history.

4. Feedback on potential citation standards.

Proper citation is essential for establishing credit for scientific work; this is inherent to the academic process. If works are not cited in bibliographies, they are less discoverable and their impact is difficult to assess by bibliometric methods.

A preprint represents an initial public communication of scientific findings and should be cited as such. In physics (and increasingly in many biology journals), preprints are cited in a manner similar to journal articles. We believe that this practice is essential for proper attribution of publicly shared ideas and information. To achieve this, preprints should be citable in all grant applications, biosketches, academic assessments, and in the main body of journal article bibliographies. However, we recommend that preprints be identified in bibliographies by a tag that clearly indicates they have yet to be peer reviewed.

In the future, preprints (pre-peer-reviewed manuscripts) and their corresponding journal articles (peer-reviewed manuscripts) should be linked together by tools (such as Crossref’s preprint service) so that their citations can be pooled. This linkage is important for maintaining an accurate and visible record of the evolution of a study. At present, interim versions of the work are easily lost or not recognized.

5. Insight on the possible need and potential impact of citing interim products on peer review of NIH applications.

Preprints will enable researchers to publicly demonstrate progress toward goals during a time when their research would otherwise be invisible during the journal review process. For this reason, citing preprints in applications and reports will allow peer reviewers to determine an applicant’s productivity with greater accuracy, particularly the most recent work of the investigator that could be the most relevant to the application under review.  For junior faculty who have started their own labs, preprints and other interim research findings could help them to provide more complete evidence of their independence after leaving their postdoctoral lab.

Allowing preprints to be cited in applications and reports would incentivize the creation of these interim research products, which we believe will significantly advance scientific progress as described in question 3.

6. Advice on how NIH reviewers might evaluate citations of interim research products in applications.

We favor the language used in the CIHR Peer Review Manual for Grant Applications, with a recommended addition (bolded):

“Communications, quick-print reports, letters and electronic distribution of pre-prints are important vehicles for disseminating research results. The data presented in [all] such contributions should be treated equally when assessing quality and impact, and reviewers should not regard certain types as “second class” or “grey literature.””

Evaluation of published papers is often strongly influenced by the journals in which they appear. Because the names of preprint servers (unlike the names of journals) bear no assumed relationship to article quality, reviewers should be encouraged to directly evaluate the content of preprints and assess the merit and relationship of the data to grant application. In general, relying more heavily on article content (rather than on journal title) will improve the quality of evaluation of all papers.  

7. Any other relevant information

To optimize the potential of preprints in the life sciences, these research objects should have maximum visibility and be easily discoverable, abide by data and quality standards, be considered as a “public good” governed by the scientific community, and be open to innovations that will advance the scientific mission.  

To achieve these aims, we proposed the creation of a Central Service that could aggregate and distribute (via an API) manuscripts from multiple sources. The Central Service would provide a single place for biologists to look for preprints, a role similar to that served by PubMed for peer-reviewed literature. All manuscripts submitted to preprint servers would ideally be formatted and licensed in a standard way so as to facilitate data mining and other applications. This will provide opportunities for innovation and resources for scientists well beyond the current state of preprints today.

Other forms of interim research products also could be deposited to the Central Service, and we recommend that all software used or created by the Central Service be open source. An international governing body should be created to oversee the operation of the Central Service, as has been implemented for other databases created for the scientific community (e.g. the protein data bank).

As preprints are a public good, we recommend that the cost for the Central Service should be borne by a consortium of funding agencies, at least for the first five years of operation until the establishment of a sustainable business model. The benefits will be large and the cost minuscule compared with publication charges being paid by funding agencies.  We encourage the NIH to participate in such a consortium and provide funding for a Central Service and its governance.

We believe that the NIH should develop clear policies on preprints/interim research products and encourage their use in biosketches, applications, reports, and post-submissions materials. These steps will incentivize universities to develop policies regarding the use and evaluation of preprints in academic appointments and promotions and in the fulfillment of criteria for awards of graduate degrees. Preprints could be used in hiring, promotion, and tenure of faculty. Additionally, preprints could fulfill PhD graduate criteria or demonstrate productivity for postdoc fellowships and job applications. Taken together, preprints could promote career transitions and reduce time to independence, a topic of great concern to the NIH and the scientific community in this era in which the increasing age of newly independent biomedical scientists is a matter of national concern.