Systematic Review: 3. Assess quality

Analysis - Step-by step

Screen

Apply criteria

Assess quality

Extract/Manage Data

Assess quality

The next step in the process is to assess the quality of the included studies.  There are a number of checklists available to assist you with this.  A checklist should be chosen to assess sources of bias that are likely to affect results in relation to the research question.  If your review is for publication or submission to a particular organisation, make sure you check if they have any requirements regarding checklist selection.

There are several different checklists or tools available to you:

     CASP checklists - 8 appraisal tools 
     JBI checklists - 13 appraisal tools 
     Dartmouth checklist  generic worksheet that can be adapted to different studies
     Consort checklist  checklist of information to include when reporting a randomised trial
     ROBINS-I  assess risk of bias in non-randomised studies of interventions
     COSMIN  consensus-based standards for the selection of health measurement instruments

     QUADAS-2  evaluate the risk of bias and applicability of primary diagnostic accuracy studies

     Cochrane handbook  section on assessing the quality of the body of evidence (The GRADE approach)

     Cochrane handbook - section on analysing and extracting data - Chapter 7 - Selecting studies

At a minimum, your systematic review should state what types of studies should be included and the minimum threshold for inclusion (e.g. RCTs with full randomisation) to reduce the likelihood of including studies with poor quality evidence.  A risk of bias assessment also helps to inform the interpretation of the review's results.

When establishing your protocol, you should have ensured you identified any thresholds for inclusion relating to the quality of evidence that would be considered.  

  • An example might be requiring randomised control trials to use full randomisation or double-blinding.

 


          Exercise 8 - Assess Quality