Skip to Main Content

Systematic Review: Develop criteria

Copy of published guide 'systematic review' (Paula Todd version) for Editing taken in July 2021

Develop criteria

An important characteristic of a systematic review is that the search is replicable; someone else should be able to follow the strategy and end up with the same articles. It’s very easy to look at an article and decide that you want to include it in your study, because the results look good or you’ve heard of the authors, or any other factors. To eliminate these personal biases as far as possible, it’s crucial to develop clear inclusion and exclusion criteria before running your final search. This criteria is usually included in your SR protocol.

Some examples of criteria you might wish to consider include:

Ideally a systematic review search strategy would be conducted without language restrictions. However, given the extra time required to obtain translations, you may decide to only include papers that you (and your co-reviewers) can read. If you make the decision to exclude based on language, best practice would be to exclude at the full-text screening stage, not before, as the abstracts are generally in English.

If you would like to be more inclusive but are not resourced to utilise translation services, then consider free services such as Google Translate. The following article discusses the accuracy of using Google Translate for this purpose:

The Accuracy of Google Translate for Abstracting Data from Non–English-Language Trials for Systematic Reviews

You need to justify any date limit included in a systematic review. For example, if your review was looking at the effectiveness of a particular drug, you might reasonably search from when the drug became approved onwards. Similarly, if an intervention has superseded a previous one, it may be appropriate to search from that time period forwards.

If you would like to limit your search to human studies only, it is best practice to identify animal-only studies first, and then exclude those from the search results.

You should not use the human limits available in databases as these will only work for indexed studies. Use an appropriate filter instead to remove animal studies whilst ensuring you don't remove studies that also included humans.

An example for Ovid Medline would be:

exp animals/ not humans.sh.

Then remove this result set from your results using NOT.

The Cochrane Handbook, Technical Supplement to Chapter 4 provides this and examples for other databases.

Depending on your research question, you may decide to limit your research to a specific age group or groups. Complexities may arise due to overlapping age categories reported in the literature, and inconsistencies in reporting or indexing.

Some MeSH headings related to age include:

  • Infant, Newborn
  • Infant
  • Child, Preschool
  • Child
  • Adolescent
  • Young Adult
  • Adult
  • Middle Aged
  • Aged
  • Aged, 80 and over

Combinations of subject headings and keywords should be used to reliably retrieve age-specific studies, including those that are not indexed. Pre-tested filters may also be useful when investigating by age.

Certain study designs will provide stronger evidence for your research question. For example, an RCT study design for a therapy/treatment question. This is often illustrated by an evidence pyramid.

Always ensure that a study design restriction is appropriate for your SR topic. For example, if looking at the effect of an intervention on elderly patients, think about how many RCTs will exclude (or include few) elderly patients. In some cases, inclusion of evidence from other study designs may be appropriate.

For more information on levels of evidence see:

Burns, P. B., Rohrich, R. J., & Chung, K. C. (2011). The levels of evidence and their role in evidence-based medicine. Plastic and reconstructive surgery, 128(1), 305–310.

Or for an example of an evidence hierarchy see:

NHMRC Evidence Hierarchy: designations of 'levels of evidence' according to type of research question