Clinician’s Corner: How might preprints benefit sharing clinical trial results?

Blog post by ASAPbio Fellow Suraj Kannan

The recent emergence of preprints in clinical research (most prominently through medRxiv) has raised a number of questions about their potential for benefit/harm. Preprints and preprint servers have existed in various basic and hard science disciplines for the past several decades, and have improved speed of dissemination and access to new findings. However, there is a general perception that the clinical research space is unique. Firstly, the pace of clinical research is thought to be slower, and the incentives may be different – in particular, with lower likelihood of inter-lab competition and risk of “scooping.” More critically, increased access to potentially erroneous results in clinical research may pose significant harm to patients. With regards to the latter set of issues, medRxiv has adopted a number of policies specifically designed to minimize harm (including screening to consider impact on public health, and visible disclaimers to facilitate proper reporting), and ongoing surveillance of clinical preprint servers will be necessary to ensure their continued safety [1]. Yet, the question of how specifically preprints can benefit clinical medicine is one that ought to be explored more thoroughly. In this post, we discuss some of these potential benefits, specifically in the context of publishing the results of clinical trials.

Speed of Dissemination

The primary purported benefit of preprints is to improve how quickly new results can be shared. In the particular case of clinical trials, publication delays can be enormously long – for example, 4-8 years from initiation of the trial [2, 3], 1-3 years from completion of the trial [4], 3-4 years from submission/presentation of an abstract at a conference [5], and 1 year from issuing a pharmaceutical press release [6]. These long timeframes inevitably prevent timely release of results that could directly impact patient health. While an extended time frame in data gathering/analysis may be inherent to the nature of clinical trials, other delays may be associated with peer review, and represent a potential area for improvement by preprints.

A common rebuttal is that long peer review times are necessary for clinical trial results. Given the potential for patient harm, the argument goes, it is better to invest time upfront to ensure rigorous review rather than prematurely report inaccurate results. Ironically, however, this may already be happening to some degree in the form of abstracts, conference presentations, and pharmaceutical press releases. Clinicians can and do change their practices in response to these pre-publication materials [7–9]. This may be particularly true for new applications of existing drugs (e.g. off-label use), which can immediately change prescribing practices [10]. Moreover, these results can indirectly affect practice as they are often included in systematic reviews and meta-analyses. However, these abbreviated reports are often incomplete or even deceptive in their description of results [11, 12]. In this context, while preprints may not guarantee completely accurate results, they at least provide clinicians with data to make important decisions. Thus, preprints can both get new information to clinicians faster while providing them useful information to guide decision-making.

Negative Results

While peer review serves a vital role in assessing clinical trial results, clinical journals are additionally subject to specific editorial policies regarding scientific prestige and/or impact, which may in turn lead to biases in article selection. In particular, it is well-known that clinical trials suffer a positive results bias, wherein trials showing “positive” results are more likely to be published and are typically published faster than those reporting negative or null findings [2, 13–19]. In many cases, conference abstracts reporting negative results never get published. This bias occurs both at the author stage, where authors may choose not to pursue the publication of negative trial results, and at the editorial stage, where negative results may be deemed to be of lower impact. Even more so than in basic science, the failure to publish negative results can lead to harm, by failing to provide clinicians with information that could guide their decision-making. Preprints may provide a solution, as considerations around impact do not guide preprint posting or dissemination. In combination with existing policies surrounding clinical trials, in particular trial registration [20], preprints can thus alleviate publication biases.

Opening Access to Data and Peer Review

Peer review may play an even more critical gatekeeping role in clinical research in general, and clinical trials in particular, compared to basic science. The main conclusions of trials are often modified or even reversed after rigorous review [21]. However, journal-based peer review is not infallible in identifying all possible mistakes. By opening up the review process to the community, preprints can enable faster correction of inaccurate results than through conventional journal-issued corrections [22]. Moreover, by enabling increased critical engagement, preprints can foster data sharing and re-analysis of trial data [23–26], rather than enshrining results only validated by a small pool of reviewers. This is further facilitated when preprint servers require data sharing/availability statements, as has been done by medRxiv. To fully reap the benefits, preprints will need to be paired with new open peer-review platforms – and indeed, several already exist or are being developed. Nevertheless, opening access to trial results will allow for more scrutiny and validation of data and allow clinicians to make data-driven decisions.

Preprints as a tool to equip clinician decision-making

Given the recency of medRxiv, it is perhaps too early to decisively track the impact of preprints on patient health. This is particularly true as the clinical preprint space has largely been dominated by the COVID-19 pandemic, and thus it is unclear what unique challenges preprints may pose in a non-pandemic context. While peer review will always be a fundamental aspect of publication, preprints can help alleviate many of the biases in clinical research. In particular, in the context of clinical trials, preprints can provide clinicians faster access to a broader range of results, thereby better equipping them in decision-making.

References

1. T Bloom. New preprint server for medical research. BMJ 2019; 365. DOI: 10.1136/bmj.l2301.
2. S Hopewell et al. Time to publication for results of clinical trials. Cochrane Database Syst Rev 2007; 2. DOI: 10.1002/14651858.MR000011.pub2.
3. Welsh et al. Age of Data at the Time of Publication of Contemporary Clinical Trials. JAMA Netw Open 2018; 1(4). DOI: 10.1001/jamanetworkopen.2018.1065.
4. Ross et al. Time to Publication Among Completed Clinical Trials. JAMA Intern Med 2013; 173 (9). DOI: 10.1001/jamainternmed.2013.136.
5. Camacho et al. Presentation and subsequent publication rates of phase I oncology clinical trials. Cancer 2005; 104(7). DOI: 10.1002/cncr.21337.
6. Qunaj et al. Delays in the Publication of Important Clinical Trial Findings in Oncology. JAMA Onc 2018; 4(7). DOI: 10.1001/jamaoncol.2018.0264.
7. Falagas et al. Clinical decision-making based on findings presented in conference abstracts: is it safe for our patients? Eur Heart J 2006; 27(17). DOI: 10.1093/eurheartj/ehl175.
8. Gross et al. Relation Between Prepublication Release of Clinical Trial Results and the Practice of Carotid Endarterectomy. JAMA 2000; 284(22). DOI:10.1001/jama.284.22.2886.
9. Kassirer et al. Prepublication Release of Journal Articles. NEJM 1997, 337. DOI: 10.1056/NEJM199712113372409.
10. Fugh-Berman et al. Off-Label Promotion, On-Target Sales. PloS Med 2008; 5(10). DOI: 10.1371/journal.pmed.0050210
11. Kuriya et al. Quality of Pharmaceutical Industry Press Releases Based on Original Research. PloS One 2008; 3(7). DOI: 10.1371/journal.pone.0002828.
12. Yavchitz et al. Misrepresentation of Randomized Controlled Trials in Press Releases and News Coverage: A Cohort Study. PLoS Med 2012; 9(9). DOI: 10.1371/journal.pmed.1001308.
13. Rowbotham et al. The case for publishing ‘negative’ clinical trials. Pain 2009; 146(3). DOI: 10.1016/j.pain.2009.09.026.
14. Hwang et al. Failure of Investigational Drugs in Late-Stage Clinical Development and Publication of Trial Results. JAMA Intern Med 2016; 176(12). DOI: 10.1001/jamainternmed.2016.6008.
15. Scherer et al. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2007; 18(2). DOI: 10.1002/14651858.MR000005.pub3.
16. Hopewell et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 2009. DOI: 10.1002/14651858.MR000006.pub3.
17. Turner et al. Selective publication of antidepressant trials and its influence on apparent efficacy. NEJM 2008; 358(3). DOI: 10.1056/NEJMsa065779.
18. Melander et al. Evidence b(i)ased medicine–selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003; 326(7400). DOI: 10.1136/bmj.326.7400.1171.
19. Song et al. The significance of the trial outcome was associated with publication rate and time to publication. J Clin Epidemiol 2017; 84. DOI: 10.1016/j.jclinepi.2017.02.009.
20. Dickersin et al. Registering clinical trials. Jama 2003; 290(4).
21. Bauchner. The Rush to Publication: An Editorial and Scientific Mistake. JAMA 2017; 318(12). DOI: 10.1001/jama.2017.11816.
22. See, for example, https://twitter.com/lteytelman/status/1262079934488383488.
23. Krumholz et al. Open Access to Clinical Trials Data. JAMA 2014; 312(10). DOI: 10.1001/jama.2014.9647.
24. Ebrahim et al. Reanalyses of randomized clinical trial data. JAMA 2014; 312(10). DOI: 10.1001/jama.2014.9646. 
25. Bauchner et al. Data Sharing: An Ethical Imperative. JAMA 2016; 315(2). DOI: 10.1001/jama.2016.2420.
26. Verma et al. Preprints: a Timely Counterbalance for Big Data–Driven Research. J Gen Intern Med 2020; 35. DOI: 10.1007/s11606-020-05746-w.