Systematic Review: Develop criteria & protocol

An important characteristic of a systematic review is that the search is replicable; someone else should be able to follow the strategy and end up with the same articles. It’s very easy to look at an article and decide that you want to include it in your study, because the results look good, you’ve heard of the authors, or any other factors. To eliminate these personal biases as far as possible, it’s crucial to develop clear inclusion and exclusion criteria at the outset for your SR protocol. It may be useful to consider core outcome measures used in research in the field that you are reviewing when developing this criteria.

Some examples of criteria you might wish to consider include:

• Language of publication:  Ideally a systematic review search strategy would be conducted without language restrictions.   Best practice (if limiting by English language) would be to exclude at the full-text screening stage. See this article for tips on Google translate.
• Year of publication:  You need to justify any date limit included in a systematic review. 
• Human studies:  If you limit your search to human studies only, it is best practice to identify animal-only studies first, and then exclude those from the search results. See Cochrane Handbook Technical Supplement to Chapter 4 (p.60) for detailed information.
• Age groups:  Combinations of subject headings and keywords should be used to reliably retrieve age-specific studies, including those that are not indexed. Pre-tested filters may also be useful when investigating by age.
• Study design:  Always ensure that a study design restriction is appropriate for your SR topic. See this article by Burns et al. (2011) on levels of evidence.

A protocol outlines the plan or set of steps to be followed. A review protocol should describe the rationale for the review, the methods that will be used to locate, select and critically appraise studies, and to analyse and synthesise data.

The registration of a protocol is designed to provide transparency and facilitate the specification of criteria and methods at the outset, rather than influenced by study findings. PRISMA-P is a useful template for developing a systematic review protocol.

Making protocols publicly available can also reduce duplication of effort. For systematic reviews, this is usually done in a registry (such as Prospero) or publishing it in a dedicated journal (such as Systematic Reviews) during the planning stages of the review.

Scoping review protocols are often loaded to OSF Registries and there is a template for writing a scoping review protocol available in OSF: Lely et. al. (2023). (The protocol template document has a template with key sections, and the protocol guidance document provides instruction for each section along with links to examples). Another protocol template is available from JBI.

For other types of reviews, you may consider making a protocol available via OSF Registries or Bridges (the Monash University Repository). Pieper and Rombey (2022) have identified further options for prospectively registering protocols.