Writing a systematic review: Part 2

Deborah Hawthorne, Amanda Quek, Kenneth Lee, Shania Liu, and Amy T. Page  

Advanced Pharmacy Australia (AdPha) proudly supports the Research Toolkit series, which aims which aims to support members in conducting and publishing their research. This series is coordinated by the AdPha Research Leadership Committee, and hence shares the insights and experience of our most research-passionate members. If you’re keen to make a difference to patient care, not just in your daily practice, but in improving practice itself, then this series is for you.

Contents:

What is a systematic review?

A systematic review is a comprehensive summary of the available literature on a topic or question of interest. The Cochrane Consumer Network uses the following definition:

“[a] systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions. Judgments may be made about the evidence and inform recommendations for healthcare”.1

Systematic reviews are extremely varied and cover a wide breadth of topics. Researchers could be using this type of review to pull together known information in a particular area of interest to try and discover previously unknown facts. Alternatively, they could be looking to bring together single studies to try and find an answer that could not be answered by looking at the studies individually. Researchers could even be attempting to find issues in current primary research that may need to be investigated further through future studies.

A systematic review aims to identify relevant studies, appraise them, and then summarise the results using a reproducible search strategy. This systematic approach sets this type of review apart from other types of literature reviews and involves documenting each step you take in the preparation and analysis of the studies you find to support the research question. For help in deciding if a systematic review is the correct approach for your query, please refer to ‘Writing a systematic review: Part 1’ by Liu et al.2

There are five essential steps in writing a systematic review; plan, search, analyse, synthesise, write, and publish. This article will delve into the initial two stages of plan and search (and document).

STEP ONE — PLAN

This first step of planning is quite involved and includes four sub-steps. These are: developing a research question by separating into concepts; performing a preliminary or scoping search; developing a plan and; deciding on specific search terms.

Separate the research question into concepts

There are two question formulation frameworks discussed here — PICO(T) and SPIDER. PICO(T) and SPIDER are mnemonic devices that can help a researcher develop their research question. PICO(T) is generally used for intervention style and quantitative queries (Box 1), while SPIDER can be helpful in qualitative searches (Box 2).

Example 1: You are interested in finding out if strength training is an effective pain relief for work related back pain. The following table uses PICO(T) to help break down the question into parts and develop the research question.

Box 1: PICO(T) Framework
Population/Problem Start with the problem (disease, condition etc.) or population (patient, characteristics etc.) Example: work related back pain
Intervention/Exposure What is the proposed intervention for the population or problem (diagnose, treat, observe etc.) Example: strength training by physiotherapist
Comparison What is the alternative to the intervention? This can be no intervention Example: rest
Outcomes What outcome do you wish to measure (mortality, morbidity, complication, treatment success etc.)? Example: pain relief
(Time) What is the timeframe of treatment or measurable outcomes? (optional) Example: 12 months

Research question: in patients with work related back pain (P), is strength training by a physiotherapist (I), compared to rest (C), more effective as pain relief over (O) a 12-month period (T)?

Example 2: You are interested in finding out what pharmacy students think about using databases. The following table uses SPIDER to develop the question parts and develop the research question.

Box 2: SPIDER Framework
Sample The group/sample size of participants (this can vary) Example: fourth year pharmacy students
Phenomenon of Interest Behaviours, interventions, and experiences Example: database use
Design How the study was designed and organised  Example: survey
Evaluation May not be as empirical as quantitative studies, could be subjective such as attitudes or views Example: experience (of using databases)
Research Type Quantitative, qualitative, or mixed method studies Example: Qualitative

Research question: A qualitative exploration of fourth year pharmacy students experience of using databases.

You’ll notice in both examples that the initial concepts were vague and unstructured. Had we started researching before applying PICO(T) or SPIDER, we would have been met with overly large results that would not have necessarily answered the question at hand as well as being time consuming to sift through.

Helpful Tip

  • Search for similar reviews to get a feel for the research topic.

Perform a preliminary or scoping search

While it may feel to some that performing a preliminary or scoping search is an easy step to miss or skip — especially when you are certain of your research question — it is considered best practice to gain familiarity with the topic of interest. By completing a quick scan of existing research, you are ensuring that the topic has not been duplicated recently, that there is enough information available to be able to perform a systematic review, and at least 2–5 appropriate articles can be found when applying your research question.

Literature databases are used to conduct a preliminary or scoping searches. Databases provide access to collections of scholarly information, unlike internet search engines, such as Google Scholar, which often give non-scholarly sources and other ‘noisy’ results like duplicates. Key biomedical literature databases include PubMed and MEDLINE. Conducting a preliminary or scoping search can be as simple as typing your research question directly into one of these databases. If there are no studies found at this stage, it may be prudent to play with your search terms to those more commonly used or recognised.

Whilst most common, PubMed and MEDLINE are not necessarily the only ones that can be used at the scoping stage. You should choose which databases to use depending on your research needs. For example, if the question is behavioural then PsychINFO may be preferred. Box 3 outlines the most commonly used biomedical databases.

Box 3: Databases available
Database Descriptions
Medline This is one of the most comprehensive and up to date databases for allied health, medicine, nursing, veterinary medicine, and pre-clinical sciences. Largest component of PubMed. It is searchable via EBSCO, PubMed, ProQuest, Web of Science and Ovid (companies that host databases) etc. amongst others
Embase A comprehensive biomedical and pharmacological database. It covers worldwide literature on health and is preferred in allied health and nursing research. Key database when considering pharmacology. Covers the same subjects as PubMed/Medline. Good coverage of alternative therapies.
ProQuest Selection of journals, magazines, and theses. It is multidisciplinary but includes non-health disciplines such as economics, business, management etc.
Web of Science Provides a scientific citation indexing service (the ability to trace the impact of articles/books/ideas to other references that have used it). It is broad ranging, including journals, books, and conference proceedings. One of the larger databases. Also indexes Medline.
SCOPUS Large citation index of scientific, technical, medical, and social science database. Indexes Medline. Preferred by researchers due to useful search features.
PsychINFO Produced by American Psychological Association (APA) and is the primary psychology database. Excellent for behavioural sciences and mental health.
CINAHL Great database for allied health and nursing topics. Important as some content cannot be found in PubMed or Medline.

Helpful Tips

  • Try a simple search in PubMed or MEDLINE. MEDLINE is normally searched as a starting point as it was one of the first established biomedical databases available. Embase can also be searched initially due to containing the content of MEDLINE plus its own unique content and has more available search terms. Using either MEDLINE or Embase is fine to begin with as both will normally be used as part of the full review.
  • PROSPERO is an international prospective register of systematic reviews in health and social care that is free to search. This is a good place to search for other systematic reviews on the topic you wish to focus on and can prevent duplication. However please note, not all reviews are listed here so further exploration using other databases will be required.
  • A preliminary search can highlight studies that have referenced articles that you want to capture in your final research search. It is a good idea to take note of these papers of interest as they can help you to finalise search topics and for you to check that your search can pick up these articles.

Develop a research plan  

After a preliminary search has been conducted, you are ready to get into the details of how you will conduct your literature search. It is important to plan this process carefully, so you don’t miss any key steps.

  1. Choose your databases: Using the list presented in Box 3, consider each database for its suitability for your research needs.  Suitability will be determined by the research question but is not always a simple decision. You may choose to search several databases to ensure adequate coverage.
  2. Determine eligible studies (inclusion and exclusion criteria). This will need to be recorded so they can be shared (prose or in a table format) in the methods section of the final report. Common inclusion and exclusion criteria include (but are not limited to); date, peer review, location, setting (community pharmacy, hospital pharmacy etc.), participants age, type of study (original work, not letters), language (English), etc. 
  3. Once databases are chosen, controlled vocabulary (this is explained below in the section ‘controlled vocabulary and free text terms’ below) can be further developed.
  4. Determine search fields (title, abstract etc.). This may vary slightly between databases.

Helpful Tips

  • Check out similar reviews and author guidelines from journals you may wish to publish in to see which databases they use or recommend. Journals may have preferred databases. There is currently no consensus regarding the number of databases required to produce a high-quality systematic review, however five databases is a good point to start
  • When deciding on which databases to select, consider your topic. For example, if you want to find research about the efficacy of a cardiovascular drug, PsychINFO would be inappropriate as the database catalogues psychological studies (see Box 3).

Decide on search terms: determine controlled vocabulary and free text terms

Once databases have been selected, the next step is to determine the terms you will use to search your chosen databases. Search terms A great place to start is by identifying some studies that you know are relevant to your search and looking at the indexing terms used in these studies to assist with determining the initial search terms. To get the most out of your search terms, it is important to understand how to use Boolean operators, controlled vocabulary, and free terms.

Boolean Operators

This initial separation will allow the use of the Boolean concept. The Boolean operators; AND, OR and/or NOT are widely used across most databases (Figure 1). These seemingly simple three words are powerful when placed between search terms to make the search more accurate — essential in a systematic review.

AND — when used between terms will search the database for articles that contain both search terms and will exclude those that only have one search or the other but not both. This narrows a search.

OR — when used between terms it will search the database for ANY article that contains either work and both words. This broadens the search.

NOT — when used between terms, e.g. A NOT B, will search for articles that have term A, but exclude the articles found if they include term B. This will also help to narrow the search.

Figure 1. Combining search terms using the Boolean operators (AND, OR, NOT)

Controlled Vocabulary and Free text terms

Controlled vocabulary involves standardised subject terms applied by indexers. It allows for the application of set words that describe a whole host of different words within the one concept. Using controlled vocabulary enables the retrieval of articles that may not have the exact word being searched in the title or abstract but is still relevant to the searcher’s needs. Different databases will apply controlled vocabulary differently (i.e. use different grouping terms) so you’ll need slightly different search terms for different databases. Examples of controlled vocabulary are MeSH (medical subject headings) in MEDLINE or EMTREE in EMBASE. MeSH are a controlled list of words that applied to articles by trained indexers.3 Some articles never get MeSH so this can’t be relied on and it is important to note that when searching for articles to use both MeSH and free text words to capture all relevant information.

Many databases do not have controlled vocabulary (Web of Science etc.) or simply their controlled vocabulary does not include all relevant terms, so it is necessary to apply free text terms to your search. These searches will only find articles that include these exact free text words. To ensure you find the most relevant articles published on the research topic, you may need to examine different spellings of the word (etc. specialised vs specialized), synonyms, various grammatical differences, and even nouns and adjectives.

There are other ways to further expand (if more information is required) or contract (if there’s an information overload) the search, however this differs from database to database. The best way to learn more about the nuances of each database is to contact academics or librarians in your chosen field.

Helpful Tips

  • Seek advice from a hospital or university librarian make sure your search strategy is a good fit for your research question. This is their area of expertise!
  • The use of tools can also assist. For example, Polyglot Search Translator (or Polyglot) is systematic review software that helps to translate searches in one database to be similar or be able to be read by another software.4 Can be found at https://sr-accelerator.com.
  • If your database uses MeSH search terms, you can either use a MeSH thesaurus (can be found at https://www.ncbi.nlm.nih.gov/mesh/), or harvest them directly from articles you may have found initially when doing your preliminary search (these are often listed at the bottom of online articles under the heading MeSH terms).

STEP TWO — Search and Document results

Now that you have your research question, have chosen suitable databases developed appropriate search terms, you are ready to conduct your literature search and develop an organised record of the results.

Apply the search in your preferred databases, in most instances begin with PubMed. The search needs to be replicable and so, search results need to be documented so that future readers will be able to understand why, how, and what you found.

To manage data, consider using citation management software and extracting the data/results by saving citation files. There are several free and paid reference management software, some may be accessible via your institution. At this stage it is helpful to extract the data/results by saving citation files. The type of file you save this data as will depend on the citation management tool you use. For example, RIS files will be suitable for Endnote, Zotero, and Mendeley. Another software tool to consider is Covidence. Covidence is an online platform that allows citation management, text screening, and data extraction.

By exporting into your preferred citation management tool, duplicates can be identified and removed (hint: remember to document the number of duplicates found as you go). This record will need to be included in the final systematic review and will allow readers to understand how you got your results and let others replicate the search you conducted.

Document the following:

  • Databases searched (including providers)
  • Each search strategy for each database
  • Record number of search hits from each database before duplicates are removed
  • Date search was conducted
  • Limits/restrictions applied with justification
  • Filters used
  • Make notes if a previous search strategy was used.

For a more in-depth reporting checklist, please visit to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) website, available at PRISMA (prisma-statement.org).

Helpful Tips

  • Remember to document as you go! It’s much harder to go back and try and remember the details later.
  • Covidence is great for de-duplication of articles and can be shared amongst the research team. It can be used throughout the systematic review process for screening of title/abstract and full text, data extraction, and quality assessment. It will help in the documentation process but be even more useful in later stages of writing systematic reviews.
  • Even if you use a tool like Covidence, make sure that you record your search elsewhere so that you have something to check it with.

What next?

After completing the initial stages of writing a systematic review, planning and searching, you are well on your way to delve into the next phase of analysing, synthesising and finally, publishing your research. Before this takes place, we recommend you register your systematic review in a registry, like PROSPERO. Clearly documenting transparent research methods and intent of a systematic review, helps to reduce the impact of author bias and facilitates future peer review.

We hope this introduction into the first phase of systematic review writing was helpful in your research quest. Look out for more articles on in-depth information on writing systematic reviews in future entries in the Research Toolkit series.

References

  1. Cochrane Consumer Network. Cochrane and systematic reviews. The Cochrane Collaboration; 2023. Available from: https://consumers.cochrane.org/cochrane-and-systematic-reviews.
  2. Liu S, Mill D, Lee K, Page AT. Writing a systematic review: Part 1. Pharmacy GRIT 2021–2022; 5: 198–203.
  3. National Library of Medicine (NLM). Welcome to Medical Subject Headings. Bethesda: NLM; 2022. Available from < https://www.nlm.nih.gov/mesh/meshhome.html>. Accessed 12 April 2023.
  4. Clark JM, Sanders S, Carter M, Honeyman D, Cleo G, Auld Y, et al. Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial. J Med Libr Assoc 2020; 108: 195–207.