Intended for healthcare professionals

Editorials

The perils of preprints

BMJ 2020; 370 doi: https://doi.org/10.1136/bmj.m3111 (Published 17 August 2020) Cite this as: BMJ 2020;370:m3111
  1. May C I van Schalkwyk, NIHR doctoral research fellow1,
  2. Thomas R Hird, research fellow2,
  3. Nason Maani, Harkness fellow in healthcare policy and practice1 3,
  4. Mark Petticrew, professor of public health1,
  5. Anna B Gilmore, professor of public health2
  1. 1Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, UK
  2. 2Tobacco Control Research Group, Department for Health, University of Bath, Bath, England, United Kingdom
  3. 3School of Public Health, Boston University, Boston, MA 02118, USA
  1. Correspondence to: May CI van Schallkwyk may.vanschalkwyk{at}lshtm.ac.uk

Their use and platforms require greater scrutiny

Preprints—manuscripts that have not undergone peer review—were first embraced in physics, catalysed by the creation in the early 1990s of arXiv.org, an open online repository for scholarly papers.1 It was not until 2013 that similar initiatives were embraced by the biological and then medical sciences,2 and novel publishing platforms continue to emerge. Some commentators believe the potential for harm is outweighed by the benefits,134 but others have raised specific concerns regarding medical preprints and mitigating the risk of harm to the public.2 These discussions need to be revisited in the context of the covid-19 pandemic, which has been accompanied by an explosion of preprint publications.

Shaping global discourse

An analysis focusing on studies estimating the R0 of SARS-CoV-2 drew attention to the powerful role of preprints in shaping global discourse about covid-19 transmissibility. While showing the benefits that preprints may confer when adopting a consensus based approach—where data is extracted from multiple studies to observe trends and obtain an average with or without the exclusion of outliers—the authors also identify risks—matters of credibility and misinformation, both intentional and unintentional5—which may be increased where there are vested interests involved.

Notably, two linked preprint publications examining the association between smoking and covid-19,67 which were widely disseminated before formal peer review, drew substantial media attention, leading to claims that smoking or nicotine—or both—are protective.8 The first6 is a small observational study with multiple biases.8 The second,7 coauthored by a researcher with historical links to the tobacco industry,8 proposes a hypothesis based on the first study, while ignoring the broader evidence base on tobacco related harms.8 While additional studies have also reported low rates of infection among smokers, most drew less, if any, media attention. Making inferences from small observational studies is premature and unreliable given limited biological plausibility, and the established evidence that smoking damages health, immune function, and increases infection risk.9 Carefully designed studies that explore the causal effects of smoking and the role of comorbidities as mediating factors are needed.10

Carl Sagan warned that “extraordinary claims require extraordinary evidence,”10 and this requirement is not met. Despite this, the “message” that smoking and nicotine may be protective went global: France had to publish a special decree to control mass purchasing of nicotine, and panic buying and an increase in smoking uptake was reported in Iran.8

The linked studies were published on Qeios, an “open science” publishing platform (www.qeios.com/publishing). The Qeios platform does not subject manuscripts to editorial oversight or a moderation process, instead relying on “open post-publication peer review.” Qeios also states that “instant publishing means more lives saved,” but provides no evidence to support this assertion; the opposite may even be true. Unlike the preprint server medRxiv, Qeios does not warn readers that content isn’t peer reviewed and should not guide practice or be taken as fact by the media. At the time these studies were published, Qeios did not mandate that authors disclose competing interests, although a disclosure policy including this as a mandatory requirement has now been introduced. Platform moderation and mandatory declaration of competing interests are recognised as central to mitigating misuse of platforms and subsequent harm to public health.21112

Scientific integrity

Such disclosure gaps render platforms vulnerable to misuse by vested interests and are concerning, given that funding sources and authors’ competing interests can result in biased research and, in this specific case, given that the tobacco industry has a history of influencing research and exploiting channels of information dissemination to amplify favourable evidence, influence public discourse, and protect its interests.13 Through elaborate strategies over many decades, the tobacco industry has sought to insert doubt about tobacco related harms into the evidence base and the mainstream media, including through the work of industry front group Associates for Research in the Science of Enjoyment.14 When part of coordinated communications campaigns, poor quality and fragmented pieces of information can achieve outsized effects.14 This is also true of preprints, where a single paper, shared widely, can undermine a much broader evidence base. In an analysis of a defence case, tobacco industry defence lawyers famously noted “a little alternative causation evidence goes a long way.”15

Publishing platforms and use of preprints require standards for maintaining scientific integrity.21112 Internationally recognised standards do not currently exist, hindering formal scrutiny. At a minimum, declarations of competing interests and funding sources should be mandatory, and platform administrators should have clear open protocols for dealing with industry funded research. These policies should be transparent to researchers, the public, and journalists, who also need to be aware of why these matters are important for assessing the integrity and reliability of preprint studies and platforms, and the implications of engagement with them. More empirical research on the harms and benefits of preprints and preprint platforms, and in what contexts, is also needed to inform mechanisms to maximise the benefits of preprint publication.

There are undoubtedly benefits to open access preprints, and publication in peer reviewed journals is no guarantee of research integrity. However, it is not a foregone conclusion that the potential benefits of preprints always outweigh the risks of harm—particularly as the risk is likely to differ for different types of preprints (reports of interventional studies compared with evidence of emerging harm, for example). Use of preprints and evolving methods of publication need caution and further scrutiny. Benefits to be realised through more timely access to evidence may be undermined by harms caused by the release, dissemination, and misuse of unreliable evidence.

Footnotes

  • Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following interests: None.

  • Funding: MvS is funded by a NIHR doctoral fellowship. TH and AG are partners in the global tobacco industry watchdog Stopping Tobacco Organizations and Products, which is funded by Bloomberg Philanthropies. NM is supported by a Harkness fellowship, funded by the Commonwealth Fund. AG and MP are co-investigators in the SPECTRUM consortium which is funded by the UK Prevention Research Partnership, a consortium of UK funders (UKRI Research Councils: Medical Research Council, Engineering and Physical Sciences Research Council, Economic and Social Research Council, and Natural Environment Research Council; charities: British Heart Foundation, Cancer Research UK, Wellcome, and the Health Foundation; government: Scottish Government Chief Scientist Office, Health and Care Research Wales, National Institute of Health Research, and Public Health Agency (NI)). The views presented here are those of the authors and should not be attributed to the above funding organisations, their directors, officers, or staff.

References