The trials and tribulations of post publication peer review; Recap from November Community Call

In our November 2023 community call, we heard from Lonni Besancon (Assistant Professor, Linkoping University) about his work exposing fraudulent and unreliable science.

Lonni began with an overview of how he started researching reliability issues and highlighted that this is not his primary role; indeed, it is a large issue in the current system that there are few full-time positions focussed on quality checking the scholarly literature. Lonni described how this quality checking was particularly important during COVID-19 pandemic and how he and others identified COVID-19 work that was potentially problematic, including by looking at the time between submission and acceptance. Lonni also drew attention to the large number of these problematic articles which were published in journals in which the authors held editorial positions. This clear conflict of interest was a key theme throughout the discussion.

Lonni pointed to two useful resources for problematic papers; retraction watch for news and tracking retracted work and PubPeer for posting comments of concern on papers. Lonni also discussed the issues with getting problematic work retracted by journals and used examples of papers he had worked to get retracted as examples. Worryingly, many of these problematic papers Lonni highlighted had very high altmetric scores and were cited/read considerably more than the average article.

During his talk, Lonni used a number of examples including one of a retraction that was an open science success story, as the authors had shared their data openly, which enabled the community to discover issues with the analysis. Lonni also provided an example where a journal was reluctant to publish a rebuttal and highlighted that this is a larger issue. In this case, the editor of the journal refused to publish the rebuttal as it “did not encourage debate in scientific discussion” by calling for the problematic article to be retracted.

In addition to identifying problematic papers based on methodology or results, Lonni highlighted that there is a large issue around inappropriate use of citations that misrepresent the literature. In particular, Lonni described his most recent work that revealed manipulation of citations by a publisher – a worrying practice if this is more widespread. Lonni summarised his talk by reinstating the importance of correcting the scientific literature and that there should be an error-checking culture. However, until we provide credit for this kind of work, that desperately needed culture wont grow. We also need to collectively combat the conflicts of interest that publishers and institutions have regarding retracting work; Lonni ended by saying that many editorial staff and publishers are not helpful in retracting bad science.

Lonni’s talk provided a great introduction to an active discussion that followed. The discussion began by focusing on the personal story Lonni had told and how much of a toll this takes. Lonni pointed out that it was tough in the beginning as he felt alone in performing this kind of work. However, as he discovered others this got better and is now a strength. Lonni stated that the death threats no longer affect him too much as he sees these as remaining online but that the professional and legal attacks are much more difficult to handle. There are concerns around whether the universities would be supportive and there is a large financial cost to the researchers who quality check the scientific literature.

When asked where we should be devoting energy and resources to correcting things, Lonni highlighted the immediacy of the COVID pandemic and the importance of focussing on research with an impact on society. Tied to this, potentially problematic papers could be flagged by reviewers (e.g. through normal peer review or through initiatives like PREreview) or on PubPeer or through screening tools like the Problematic paper screening tool which looks for “tortured phrases” in papers.

One participant posed that a potential solution may be in micropublications which may help increase the chances of researchers detecting issues before a full paper is published. Additionally, providing much higher resolution images would reduce the potential for problematic papers.

As researchers, how do we feel about the involvement of the legal system? Publishers and data sleuths often face threats of legal action for retracting work which is causing irreparable harm to correcting the scientific literature. Is the answer in micropublication and producing higher resolution images with a credited researcher? Or does academia need a regulatory body, as so many other industries have had for quite some time?

If you missed the conversation you can catch a recording here https://www.youtube.com/watch?v=gECAVqafQgg.