Towards evidence based research
BMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i5440 (Published 21 October 2016) Cite this as: BMJ 2016;355:i5440
Chinese translation
该文章的中文翻译
- Hans Lund, professor1 2,
- Klara Brunnhuber, product manager3,
- Carsten Juhl, associate professor1 4,
- Karen Robinson, associate professor5,
- Marlies Leenaars, associate professor6,
- Bertil F Dorch, director7,
- Gro Jamtvedt, dean2 8,
- Monica W Nortvedt, dean2,
- Robin Christensen, professor9,
- Iain Chalmers, coordinator10
- 1SEARCH Research Group, Department of Sports Sciences and Clinical Biomechanics, University of Southern Denmark, 55 DK 5230 Odense, Denmark
- 2Centre for Evidence-Based Practice, Bergen University College, Bergen, Norway
- 3Evidence Centre, BMJ, London, UK
- 4Department of Rehabilitation, Copenhagen University Hospital, Herlev and Gentofte, Denmark
- 5Johns Hopkins University, Baltimore, MD, USA
- 6SYRCLE, Central Animal Laboratory, Radboud University Medical Center, Nijmegen, the Netherlands
- 7University Library of Southern Denmark, University of Southern Denmark, Odense, Denmark
- 8Department for Evidence Synthesis, Norwegian Knowledge Center for the Health Services, Oslo, Norway
- 9Musculoskeletal Statistics Unit, Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark
- 10James Lind Initiative, Oxford, UK
- Correspondence to: H Lund hlund{at}health.sdu.dk
Whether or not today’s medical researchers, like Isaac Newton, see themselves as “standing on the shoulders of giants,” they might still be expected to build systematically on previous research when planning new studies. Even though this issue was highlighted as early as 2005,1 2 numerous studies indicate that researchers do not use a systematic methodology to identify and refer to earlier research when justifying, designing, or discussing new research.3 4 5 6 7 8 9 10 11 This is true, even in high quality clinical studies published in the most prestigious medical journals.4 5 8 12 Rather, medical researchers select studies to cite based primarily on preferences and strategic considerations.13 14 15 16 17 18 The term “evidence based research” was coined in 2009 to indicate the approach that is needed to reduce this practice, which is an important source of research waste19 and risks unnecessary harm for patients and study participants.
In view of the easy access to both electronic research databases and high quality systematic reviews—spearheaded by groups such as the Cochrane Collaboration, and numerous evidence synthesis centres worldwide—there is little excuse for researchers failing to refer to current systematic assessments of previous research. Nevertheless, authors seem to get away with being very selective,13 14 preferentially citing studies with results that support the intervention they are evaluating.15 16 17 18 Some research funders have already taken action. For example, the National Institute for Health Research in England now requires that applicants for primary research funding justify any proposed research by referencing a current systematic review of relevant existing research to show that they have taken account of the knowledge from previous studies.20 However, much remains to be done.
It was against this background that a group of researchers decided to initiate an international network (EBRNetwork, http://ebrnetwork.org) to raise awareness of and confront this problem within health research. The network aims to reduce waste in research by promoting no new studies without systematic review of existing evidence and the efficient production, updating, and dissemination of systematic reviews.
No new studies without systematic review of existing evidence
The methodology for systematic reviews requires predefined research questions, inclusion criteria, search methods, selection procedures, quality assessment, data extraction, and data analysis, with no relevant studies excluded without explanation and the results of each study contributing to the review’s conclusions.21 22 Basic scientific training for researchers must include an understanding of the need for a systematic review of the existing evidence21 23 and the skills to critically assess, interpret, and use these reviews. For scientific, ethical, and economic reasons, current high quality systematic reviews need to be seen as an essential component of decisions about whether further studies are justified, the design of new studies, and the interpretation of new study results.1 2 17 24
Figure 1⇓ shows the process of evidence based research, from initial research question to the final decision to proceed using an optimal study design. The proposed process aims to support researchers, not to constrain them or pretend to provide all the answers: a systematic analysis is only one of many considerations when framing a research question. Researchers may draw on clinical experience, innovation, and serendipity—similar to the large range of considerations during a journal’s editorial review process.25
The number of published systematic reviews has increased steeply over recent years26 so (in principle) researchers in most areas should be able to identify at least one relevant review. However, even if systematic reviews are available, researchers still face several challenges. For example, the identified systematic reviews may be out of date or of inadequate quality, necessitating additional work (requiring relevant skills, time, and resources) before the primary study. Alternatively, a lack of common measures and definitions in included studies may preclude a statistical synthesis of results, making it difficult to integrate new results.
Taking account of relevant ongoing studies when preparing systematic reviews and planning new studies presents a further challenge for evidence based research. Although registration of new trials has now become standard,27 there has been less progress in the registration of other types of study.
Efficient production, updating, and dissemination
Conducting systematic reviews is resource and time consuming, and many clinical researchers are not trained to do it. Integration of systematic reviews within the research context not only challenges researchers to acquire the required skills but to make sure that any attempts to accelerate the underlying processes do not impair quality. This necessitates changes to training curriculums; close collaboration between researchers and librarians, information specialists, and IT experts; and focused investment to optimise and automate the processes.
Various initiatives in recent years have started to tackle the problem. Most prominently, a series of articles dealing with how to avoid waste in research,28 29 30 31 32 33 34 led to the REWARD (Reduce Waste and Reward Diligence) initiative (http://researchwaste.net/). Other important initiatives have been launched to prepare, update, and disseminate systematic reviews more efficiently, such as the Cochrane Collaboration.35 The realisation that many of the tasks of preparing a systematic review could be automated led to the International Collaboration for the Automation of Systematic Reviews being established in 2015. New ways to prepare and update systematic reviews have been developed, such as “living systematic reviews” 36 37. Cumulative meta-analyses and other methods have been used to identify whether new research is needed.38 39 40 41 Of particular interest to researchers planning to conduct a systematic review are new user friendly software solutions (http://systematicreviewtools.com/). To avoid duplication of effort and ensure all reviews are published, it is now possible to register systematic reviews in PROSPERO,42 with another registry dedicated to systematic reviews of animal studies (http://www.syrcle.nl/).
Responsibilities for evidence based research
Our evidence based research statement sets out the responsibilities of everyone involved in research (box). As shown in figure 1⇑, the researcher’s responsibility is to plan and conduct new research informed by all former and ongoing research relevant to the proposed new research. It is unnecessary to prepare a new systematic review if adequate reviews are already available. All researchers should be able to search for, critically appraise, and interpret systematic reviews in the context of new study results. Conducting or updating a systematic review can be outsourced, although it has been suggested that all health researchers should begin their training by preparing at least one systematic review to ensure sufficient understanding.23
The evidence based research statement
To embark on research when there are no systematic reviews showing that a genuine uncertainty exists, particularly when the research involves people and animals, is unethical, unscientific, and wasteful. Researchers, research funders, research regulators and research ethics committees/institutional review boards, publishers of research, research institutions/educators, and information specialists often fail to use earlier research systematically when preparing to initiate, fund, or publish the results of new studies.
Below we set out stakeholders’ responsibilities to meet the aims of evidence based research:
• No new studies without adequate systematic review of existing evidence showing new research is justified, and
• Efficient production, updating, and accessibility of systematic reviews
Aim 1: No new studies without adequate systematic review of existing evidence showing new research is justified
Researchers
To prioritise research questions after taking systematic account of the totality of relevant earlier research and ongoing research
To know how to search efficiently for relevant systematic reviews and ongoing studies. If the search indicates that there are no relevant, up-to-date systematic reviews, researchers should be aware of the options for preparing or updating the review needed
To be able to assess the risk of bias in systematic reviews
To be able to supervise students studying for higher degrees in using and preparing systematic reviews
Funding agencies
To evaluate whether applicants for funds have used systematic reviews of prior research to identify and help to prioritise research questions or agendas
To evaluate whether applicants have demonstrated adequate support for their proposed research by reference to systematic reviews of prior research
To evaluate whether the designs of proposed new studies have been informed by systematic reviews of prior research
Research regulators, including research ethics committees/institutional review boards
To evaluate whether applicants have shown adequate support for their research questions by reference to systematic reviews of prior research. This expectation extends beyond randomised trials
To evaluate whether the designs of proposed new studies have been informed by reference to systematic reviews of prior research
Editors and reviewers
To assess whether the rationale and design of studies are adequately described within the context of systematic reviews of prior research
To evaluate whether description of earlier research is sufficient to enable interpretation of the results of submitted studies within the totality of relevant evidence
To evaluate whether proposals for further research take account of earlier and ongoing research
To evaluate whether proposals for further research include clear descriptions of target populations, interventions, comparisons, outcome measures, and study types
Educators
To teach the importance of an unbiased approach to knowledge synthesis (systematic review)
To teach how to seek or prepare and use systematic reviews when planning and interpreting additional research
Patients and consumers
Before agreeing to participate in research, patients should demand that research projects have been informed by systematic review of what is already known
Aim 2: Efficient production, updating, and accessibility of systematic reviews
Systematic review specialists
To participate in research and developmental activities to:
Improve the preparation and updating of systematic reviews
Develop automation of the preparation of systematic reviews
Develop tools for preparing systematic reviews more efficiently
Information specialists and librarians
To help develop methods to increase the quality and currency of literature searches
To participate in teaching researchers how to perform high quality searches for relevant studies
To participate in research and developmental activities to:
Improve the production and updating of systematic reviews
Develop automation of the preparation of systematic reviews
Develop tools for preparing systematic reviews more efficiently
Information technologists, programmers, and artificial intelligence engineers
To participate in research and developmental activities to:
Improve the production and updating of systematic reviews
Develop automation of the preparation of systematic reviews
Develop tools for preparing systematic reviews more efficiently
Funding agencies
To support development and research to:
Improve the production and updating of systematic reviews
Develop automation of the preparation of systematic reviews
Develop tools for preparing systematic reviews more efficiently
Recommendations
Instructions for authors should include requirements for evidence justifying the research for which publication is sought
Systematic reviews should be recognised as research in their own right, comparable with other types of research activity
A clear definition of a high quality systematic review should be prepared, agreed, and promoted
It is crucial that not only researchers but all other key stakeholders in the scientific process—such as patients, research funders and regulators, ethical committees, and publishers—acknowledge their responsibility for evidence based research. Research waste and unnecessary harm to patients can be prevented by granting approval only to proposals informed by a current systematic review. Demanding references to systematic reviews that synthesise all relevant earlier studies needs to become a guiding principle for all those safeguarding the research process up to publication.
Senior researchers and educators need to ensure that new researchers are taught how to conduct research that is evidence based.23 Information specialists and librarians have a crucial role in teaching students to perform a systematic search for systematic reviews, and in preparing and publishing research about improved ways to perform literature searches.
Last but not least, research and adequate investment in the automation of systematic reviewing is crucial to achieve and maintain evidence based research, as the growing size of the healthcare literature and increasing complexity of studied interventions 26 make it impossible to keep up with the workload using traditional manual methods of reviewing.
Invitation to action
The task of identifying and addressing the challenges for all stakeholders can only be confronted efficiently through international collaboration. The EBRNetwork has prepared a road map for publications (including a systematic review into the current status and effects of evidence based research itself) to raise awareness of the challenges for different stakeholder groups (implication papers) and to suggest how to deal with the tasks identified in or implied by the evidence based research statement (how-to papers). For more details, see ebrnetwork.org.
Key messages
Embarking on research without reviewing systematically what is already known, particularly when the research involves people or animals, is unethical, unscientific, and wasteful
A systematic review of relevant evidence can establish whether the proposed research is truly needed
Some research funders now require applicants to refer to a systematic review of existing research
Research waste can also be reduced by efficient production, updating, and dissemination of systematic reviews
Footnotes
We thank participants in the Bergen meeting and members of the EBRNetwork for their help in creating this statement. We are also grateful for financial support from the Research Council of Norway, Bergen University College, Norwegian Knowledge Centre for the Health Services, the Musculoskeletal Statistics Unit, the Parker Institute (supported by the Oak Foundation), and the research group SEARCH (Synthesis of Evidence and Research) at the University of Southern Denmark, for making this statement and the EBRNetwork a reality.
Contributors and sources: This statement arose from an inaugural meeting of the EBRNetwork in Bergen, December 2014, arranged by MWN, GJ, and HL. All authors participated in the meeting and in the development of the EBR statement.
Competing interests: We have read and understood BMJ policy on declaration of interests and have no relevant interests to declare. The EBRNetwork does not accept support from health technology companies.
Provenance and peer review: Not commissioned; externally peer reviewed.