We all want to inform our decision making with evidence, but filtering out the pseudoscience can be quite a feat. Especially since it is so often easily digestible and fits nicely with an existing point of view. What we need is something that will synthesise the evidence for us... By Penny Presta.
Systematic reviews were devised in 1979 by Archie Cochrane in an attempt to overcome such challenges by providing ‘critical summaries’ for the medical profession . A systematic review implements a standardised approach to gathering evidence relating to a specific research question from an exhaustive set of studies, and to analysing the data in context to assess the strength of the evidence. The quality of systematic reviews varies, although Cochrane Reviews use rigorous scientific methods and are sometimes considered to be the ‘gold standard’. Standards have also been set by PRISMA and the Institute of Medicine.
The Australian Government funds free public access to the Cochrane Library through a national subscription. This gives Australians a unique level of access to quality evidence to inform their healthcare choices .
Prior to undertaking a systematic review a ‘protocol’ is published outlining in detail the planned methods that will be used. By searching existing protocols you can ascertain that no-one is currently doing the research you wish to do and that a new review is required. Further, by stating your aims and study design in your protocol, you are reducing the risk of bias that may arise if you were to formulate the inclusion criteria for studies after seeing the results.
Anatomy of a systematic review
- A clear clinical question is formulated
- Inclusion and exclusion criteria are explicitly defined
- A structured search strategy is developed and an extensive search for studies is conducted
- Studies are selected based on pre-defined criteria
- Quality of primary studies is assessed based on pre-defined criteria
- Extraction of relevant data
- Synthesis of data (possible meta-analysis)
- Findings are reported/disseminated
Systematic reviews have some unique features that make them differ from standard literature reviews. Below are some requirements of Cochrane systematic reviews.
- Must be updated every two years or include an explanation as to why this hasn’t happened.
- Should have more than one author. This is effective in reducing potential author bias in selection of studies and data extraction, and to help detect any errors.
- Can be replicated (and therefore verified) due to the comprehensive documentation of the search and selection methodologies used.
- Poor quality studies are eliminated (via pre-defined exclusion criteria) even when there are few other studies available. This can provide clarity in areas previously thought to show opposing conclusions.
- Where possible, an international perspective is taken and results considered in a broad context.
A meta-analysis is a statistical technique that combines the findings of relevant studies and analyses the resulting data set. Some systematic reviews will include a meta-analysis when assessing the effectiveness of a healthcare intervention.
Using Systematic Reviews
As a student you may be asked to use primary research articles to answer your research question. A systematic review is a secondary source so would not suit this purpose. However, if you find a relevant systematic review, looking at the reference list will give you a ready-made list of primary research articles on the topic! And of course, you may also use systematic reviews to guide your own health care decisions.
Systematic reviews can be found in the Cochrane Library, as well as in databases such as EBM Reviews, PLoS and the Campbell Collaboration. For assistance finding or conducting your own Systematic Reviews contact your subject librarian.
- Cochrane, A. L. (1979). 1931-1971: A critical review, with particular reference to the medical profession. In G. Teeling-Smith & N. Wells (Eds.). Medicines for the year 2000 (pp. 1-11). London: Office of Health Economics.
- Department of Health. (2014). National access. Retrieved from http://www.health.gov.au/internet/main/publishing.nsf/Content/health-cochrane.htm