Complexity in a systematic review may arise for a number of reasons. The topic itself may be complex with multiple, and poorly understood, interactions between intervention, mediators, and outcomes. There may be high levels of heterogeneity making it difficult to synthesise the data from the included studies. Heterogeneity may be statistical, or due to methodological characteristics of the included studies, or there may be conceptual heterogeneity for example diversity in the interventions, populations, or outcomes in the included studies. Methodological developments, such as network meta-analysis and meta-regression, have increased the potential for statistical synthesis to be used in complex systematic reviews. However, there are many instances where heterogeneity means that statistical pooling is not appropriate meaning that quantitative data have to be synthesised narratively.
Narrative synthesis is a generic term and there are many other terms used to describe approaches which rely on narrative, rather than statistical, synthesis. For example, thematic synthesis, realist synthesis, meta-ethnography, framework synthesis etc. Some of these approaches are developed specifically for synthesis of qualitative data. As such these reviews have a different, if often complementary, focus to reviews aiming to establish an estimate of effect size. In addition to developments to improve meta-analytical methods to handle heterogeneity, there have also been valuable developments to promote a methodology for the synthesis of qualitative data. By comparison there is little guidance on narrative synthesis of quantitative data despite it being a frequently used method. A common criticism of narrative synthesis is that it is difficult to maintain transparency in the interpretation of the data and development of conclusions. This ultimately threatens the value of the synthesis and the extent to which the conclusions can be relied upon.
Narrative reviews are also used to synthesise ideas and theories. In these cases narrative approaches are used not because meta-analysis is impractical for the reasons given above but because the object of interest (the intervention) has not been sufficiently understood. The role of the narrative review in these circumstances is to synthesise the multiple purposes and theories about how and why an intervention might work or not in different circumstances. This kind of review is useful for scoping the landscape in relation to particular interventions.
Below is an introductory list of resources which may be useful if you are conducting a review where the data cannot be synthesised statistically. It is important to note that authors writing about narrative approaches to synthesising the literature have different views about the best way to conduct such reviews – all argue for robust approaches but the nature of these approaches vary.
Main website for realist methods
Center for Advancement in Realist Evaluation and Synthesis (CARES) https://realistmethodology-cares.org/
Resources for mixed studies reviews
Wiki Toolkit for Mixed Studies Reviews
Guidance on realist synthesis & RAMESES (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) http://www.ramesesproject.org/
Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology. 2012;12(1):181. http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-12-181
Bibliography including methods guidance and tools
Anderson L, Oliver S, Michie S, Rehfuess E, Noyes E, Shemilt I. Investigating complexity in systematic reviews of interventions by using a spectrum of methods.Journal of Clinical Epidemiology.0; http://dx.doi.org/10.1016/j.jclinepi.2013.06.014
Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, et al. Using logic models to capture complexity in systematic reviews. Research Synthesis Methods. 2011;2(1):33-42. http://onlinelibrary.wiley.com/doi/10.1002/jrsm.32/abstract
Boaz A et al (2006) ‘A multitude of syntheses: a comparison of five approaches from diverse policy fields’. Evidence & Policy 4(2) 2006
Mozygemba K, Refolo P, Sacchini D, Tummers M, Rehfuess E. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. http://www.integrate-hta.eu/wp-content/uploads/2016/02/Guidance-on-choosing-qualitative-evidence-synthesis-methods-for-use-in-HTA-of-complex-interventions.pdf
Burford B, Lewin S, Welch V, Rehfuess E, Waters E. Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making.Journal of Clinical Epidemiology.0; http://dx.doi.org/10.1016/j.jclinepi.2013.06.017
Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Systematic Reviews 2012;1(1):28.
Gough D, Oliver S, Thomas J, editors. An introduction to systematic reviews. London: Sage, 2012.
Ioannidis, J. P. A., N. A. Patsopoulos, et al. (2008). Reasons or excuses for avoiding meta-analysis in forest plots. British Medical Journal336(7658): 1413-1415.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009;339(jul21_1):b2700-.
Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health 2005;59(9):804-08.
Noyes J, Hendry M, Booth A, Chandler J, Lewin S, Glenton C, et al. Current use was established and Cochrane guidance on selection of social theories for systematic reviews of complex interventions was developed. Journal of Clinical Epidemiology. http://www.jclinepi.com/article/S0895-4356%2816%2900005-6/abstract
Theory in complex reviews Wiki: http://theoryinreviews.pbworks.com/w/page/85156132/FrontPage
Pawson R. et al. (2005) ‘Realist review – a new method of systematic review designed for complex policy interventions’. Journal of Halth Services Research & Policy 10 (1): 21-34 doi: 10.1258/1355819054308530
Petticrew M. Why certain systematic reviews reach uncertain conclusions. British Medical Journal 2003;326:756-8.
Petticrew M. Presumed innocent: Why we need systematic reviews of social policies. American Journal of Preventive Medicine 2003;24(1):2-3.
Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ 2001;322(7278):98-101.
Petticrew M, Roberts H (2006) Systematic Reviews in the Social Sciences: A practical guide. Oxford: Blackwell Publishing.
Petticrew M, Anderson L, Elder R, Grimshaw J, Hopkins D, Hahn R, Krause L, Kristjansson E, Mercer S, Sipe T, Tugwell P, Ueffing E, Waters E, Welch V.Complex interventions and their implications for systematic reviews: a pragmatic approach.Journal of Clinical Epidemiology.0; http://dx.doi.org/10.1016/j.jclinepi.2013.06.004
Pigott T, Shepperd S. Identifying, documenting, and examining heterogeneity in systematic reviews of complex interventions.Journal of Clinical Epidemiology.0; http://dx.doi.org/10.1016/j.jclinepi.2013.06.013
Popay J, Roberts H , Sowden A , Petticrew M , Arai L, Rodgers M , Britten N, Roen K, Duffy S. Guidance on the conduct of narrative synthesis in systematic reviews: a product of the ESRC methods programme . Lancaster, 2006. http://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications.php
Pope, C., N. Mays and J. Popay (2007). Synthesizing qualitative and quantitative health evidence, Open University Press.
Thomson H and Thomas S. 2013.The effect direction plot: visual display of non-standardised effects across multiple outcome domains.Research Synthesis Methods.4; 1; 95-10110.1002/jrsm.1060. http://onlinelibrary.wiley.com/doi/10.1002/jrsm.1060/abstract
Thomson, H. (2013). "Improving Utility of Evidence Synthesis for Healthy Public Policy: the Three Rs (Relevance, Rigor, and Readability [and Resources])." American Journal of Public Health 103(8): e17-e23.
Squires J E, Valentine J C & Grimshaw J. Systematic reviews of complex interventions: framing the review question.Journal of Clinical Epidemiology.0; http://dx.doi.org/10.1016/j.jclinepi.2013.05.013