Original ArticleAdvances in the GRADE approach to rate the certainty in estimates from a network meta-analysis
Introduction
In 2014, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group presented guidance to evaluate the certainty of the evidence (confidence in evidence, quality of evidence) from network meta-analysis (NMA). [1] This guidance represented a response to the need for establishing the certainty of the evidence for each paired comparison within an NMA and the desirability of implementing widely used GRADE criteria [2] to inform those judgments [3].
The application of GRADE's approach to rate the certainty of the evidence in an NMA may appear onerous in networks with many interventions. While in the simplest network with only three treatments (e.g., treatments A, B, and C), researchers must undertake the certainty assessment three times (i.e., they must address the direct, indirect, and network estimates for A vs. B, A vs. C, and B vs. C), the requirement in a network with 6 treatments is 15 assessments and in a network with 12 treatments 66 assessments. Moreover, the assessment requires repetition for each outcome of interest.
The application of the GRADE approach to a number of NMAs [4], [5], [6], [7], [8] in the 3 years since the original guidance publication has led to advances that have strengthened the conceptual basis, dealt with challenges that have arisen in applying the approach, and—most relevant to the volume of work required in applying the GRADE approach—may make the process of assessing the certainty of the evidence more efficient. In this article, we describe these advances and their rationale and provide illustrative examples. We focus on guidance for systematic reviewers who aim to rate the certainty of the evidence of all the pairwise comparisons from an NMA, regardless of whether there is high certainty from traditional direct comparisons to inform clinical decision-making. The discussion assumes a familiarity with the basic concepts of indirect evidence and NMA and with the GRADE approach to rating certainty of evidence for bodies of evidence in conventional paired comparison meta-analysis and is restricted to NMAs of randomized trials. This article describes official guidance from the GRADE working group.
Section snippets
Assessing the certainty of the evidence from NMA—the GRADE approach
To assess the certainty associated with evidence from NMA, we must consider all the contributing evidence. This includes evidence from trials directly comparing any two interventions of interest—direct evidence—and the evidence from trials that inform an indirect comparison of the two interventions through one or more common comparators—indirect evidence. According to the GRADE approach, the certainty of the evidence from a conventional meta-analysis of randomized trials comparing two
Conceptual advances in the assessment of the certainty of the evidence from NMA
Herein, we describe two modifications of GRADE guidance that may enhance efficiency of the GRADE process. One relates to assessment of imprecision and the other to the possibility of omitting rating of the certainty of the indirect evidence (Table 1).
Application of these advances to another example
We applied the principles described herein to the NMA assessing the effects of resuscitative fluids on mortality in patients with sepsis [4]. Authors included 14 randomized trials that compared albumin, crystalloid, hydroxyl-ethyl starch, and gelatin to one another (Fig. 5). We used the judgments made by the authors in the original evaluation of the certainty of the evidence for all the GRADE domains. Table 3 presents the results.
Table 4 shows details of the assessments. The only domain for
Discussion
In this article, we have described and illustrated recent conceptual advances in the GRADE approach for assessing the certainty of the estimates from NMA. The main challenge that reviewers face when rating the certainty of the estimates of effect from NMA is the burden associated with the task. In the original GRADE guidance, for each pairwise comparison, rating a network estimate required an assessment of all five GRADE domains associated with rating down, and for all direct and indirect
References (16)
- et al.
GRADE guidelines: 3. Rating the quality of evidence
J Clin Epidemiol
(2011) - et al.
GRADE guidelines 6. Rating the quality of evidence–imprecision
J Clin Epidemiol
(2011) - et al.
Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis
Lancet
(2009) - et al.
A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis
BMJ
(2014) - et al.
The attractiveness of network meta-analysis: a comprehensive systematic and narrative review
Heart Lung Vessel
(2015) - et al.
Fluid resuscitation in sepsis: a systematic review and network meta-analysis
Ann Intern Med
(2014) - et al.
Treatment of idiopathic pulmonary fibrosis: a network meta-analysis
BMC Med
(2016) - et al.
Effects of different phosphate lowering strategies in patients with CKD on laboratory outcomes: a systematic review and NMA
PLoS One
(2017)
Cited by (428)
Optimal timing for early mobilization initiatives in intensive care unit patients: A systematic review and network meta-analysis
2024, Intensive and Critical Care NursingPreventing postpartum hemorrhage: A network meta-analysis on routes of administration of uterotonics
2024, European Journal of Obstetrics and Gynecology and Reproductive BiologyNetwork meta-analysis: The way forward for evidence-based decisions
2024, Clinical Epidemiology and Global Health
Conflict of interest: None.