Skip to Main Content

Systematic Reviews: Appraisal

What is Critical Appraisal?

Critical appraisal is a critical evaluation of research literature. It is essential for informed decision-making.

Critical appraisal evaluates:

  • Credibility - author qualifications, reputability (peer-review)
  • Accuracy - currency of the evidence
  • Reasonability - objective, unbiased (research question, study design, funding, publishing), transparency of methodologies
  • Support - use of references

Image taken from AIChE webinar:

Critical Appraisal Guide

Critical Appraisal Websites

Assess the quality of your included studies

Cochrane uses the GRADE system to rate the certainty of the body of evidence and to grade the strength of recommendations. More information can be found in the GRADE guidelines, on the Melbourne Grade Centre website, or see the grade guidelines guidance. For systematic reviews, GRADE provides a standardised approach to the appraisal of discrete outcomes from your meta-analysis or meta-synthesis.

Cochrane recommends the following checklists for assessing the risk of bias of the individual studies included in your review, as these checklists work well with the GRADE system:

  • Randomised controlled trials: RoB 2.0
  • Non-randomised controlled trials: NewCastle-Ottawa scale or ROBINS-I. (ROBINS-I is quite complex so an expert on the team is desirable for its use).
  • Diagnostic studies: QUADAS-2
  • Prognostic studies: QUIPS

Following checklists from a range of organisations are available for assessing a variety of study designs:

Critically Appraising for Antiracism Quality Appraisal Tool

Critical appraisal tools do not consider the potential impact of racial biases on a paper's quality. This supplementary tool has been developed in order to support appraisers in explicitly addressing racial bias.

Aboriginal and Torres Strait Islander Quality Appraisal Tool

The Aboriginal and Torres Strait Islander Quality Appraisal Tool consists of 14 questions that assess the quality of health research from an Aboriginal and Torres Strait Islander perspective.

Acknowledgement - much of the content in this guide has been adapted / copied from Home - Systematic Review - Subject guides at Monash University