Table 1

Comparison of the AMSTAR-2 and ROBIS tools

Name of toolConcept measuredAreas of researchNumber of itemsItems unique to tool (and associated item number)Type of overall ratingsTime (range across three studies*)
AMSTAR-2Methodological qualityIntervention16 items
  • Whether the authors explained why they selected the study designs for inclusion in the review (item 4)

  • Whether authors provided a list of excluded studies and justification for exclusions (item 7)

  • Whether authors reported on sources of funding for included studies (item 10)

  • Whether authors reported their own potential competing interests (item 16)

Overall confidence in the results of the review (critically low, low, moderate, or high), based on assessments of critical domains, predetermined by user (appraisers can alternatively use the critical domains suggested by the AMSTAR-2 developers)12-18 min
ROBISRisk of biasIntervention, diagnosis, prognosis, and biological cause19 items organised into four domains
  • Applicability of the review question to the assessors' research question (ie, do they match) (phase 1)

  • Whether eligibility criteria were appropriate for research question (item 1.2)

  • Whether restrictions in eligibility criteria based on study characteristics (eg, date, sample size, study quality, outcomes measured) were appropriate (item 1.4)

  • Whether restrictions on eligibility criteria (eg, publication status or format, language, availability of data) were appropriate (item 1.5)

  • Whether search restrictions based on date, publication format, or language were appropriate (item 2.4)

  • Whether all relevant study results were collected for use in the synthesis (item 3.3)

  • Whether studies were selectively missing from the synthesis (item 4.1)

  • Whether all predefined analyses were reported (item 4.2)

  • Whether authors acknowledged the limitations (and biases) when interpreting the results of their review (phase 3)

  • Whether authors avoided emphasising results on the basis of their statistical significance? (phase 3)

Risk of bias ratings assigned for each domain and overall (low, high, or unclear)16-24 min
  • *Perry et al18 had a slightly lower time for AMSTAR-2 than for ROBIS (14 min v 24 min); Pieper et al21 had a mean time for scoring AMSTAR-2 slightly higher than that for ROBIS (18 min v 16 min); and Gates20 had lower time for AMSTAR-2 than for ROBIS (12 min v 17 min).