Skip to main content
Effective Health Care Program

Practical Tools and Guidance for Systematic Review of Complex Interventions

Abstract

Patients, clinicians, health systems, and others want evidence to inform their health care decisions. However, health care interventions have become increasingly complex, with multiple components that address many parts of a given clinical problem at once. The effectiveness of complex interventions is influenced by complex interplays of individual characteristics, social determinants, elements of the health care delivery system, and the interventions themselves (Box 1). The complexity of these multicomponent interventions raises special challenges when interpreting and summarizing the results. Previous papers on systematic reviews of complex interventions1,2 noted inconsistent reporting among individual studies and reviews. These inconsistencies may limit the usefulness and hinder implementation of findings by end-users.

The Agency for Healthcare Research and Quality (AHRQ), under the aegis of the U.S. Department of Health and Human Services, supports the conduct of systematic reviews on a wide array of clinical and healthcare delivery topics. To address the unique challenges posed by complex interventions, the Agency provides tools that support the conduct of systematic review of complex interventions.

In June 2015, AHRQ supported a meeting of 57 experts in complex interventions to identify and develop practical tools and guidance for how to scope reviews, create appropriate analytic frameworks, and analyze data on complex interventions. Subsequent workgroups addressed key meeting themes. A steering committee provided oversight and coordination to ensure consistency and cohesiveness.

The final products, the AHRQ Series on Complex Intervention Systematic Reviews, are published in the Journal of Clinical Epidemiology. These seven papers are intended to help researchers design and conduct systematic reviews to better answer important clinical, policy, and research questions to improve health care and health outcomes.

Box 1: Definition of Complex Interventions


(adapted from Guise et al, AHRQ Series on Complex Intervention Systematic Review-Paper 1)


All complex interventions have two common characteristics:

  • they have multiple components (intervention complexity)
  • and complicated/multiple causal pathways, feedback loops, synergies, and/or mediators and moderators of effect (pathway complexity).

Additionally, they may also have one or more of the following characteristics:

  • target multiple participants, groups, or organizational levels (population complexity);
  • require multifaceted adoption, uptake, or integration strategies (implementation complexity);
  • or work in a dynamic multi-dimensional environment (contextual complexity).

AHRQ Series on Complex Intervention Systematic Reviews

Paper 1: An introduction to a series of papers that provide guidance and tools for reviews of complex interventions

Guise J-M, Chang C, Butler ME, Viswanathan M, Tugwell P. AHRQ Series on Complex Intervention Systematic Reviews – Paper 1: An introduction to a series of papers that provide guidance and tools for reviews of complex interventions. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.011.

http://www.jclinepi.com/article/S0895-4356(17)30630-3/fulltext

This introductory paper provides background about the development of the series, introduces key concepts, and provides an overview of the series scope.

Paper 2: Defining Complexity, Formulating Scope and Questions

Kelly MP, Noyes J, Kane RL, Chang C, Uhl S, Robinson KA, Springs S, Butler ME, Guise J-M. AHRQ Series on Complex Intervention Systematic Reviews – Paper 2: Defining Complexity, Formulating Scope and Questions. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.012.

http://www.jclinepi.com/article/S0895-4356(17)30631-5/fulltext

This paper describes the methodological, practical, and philosophical challenges and potential approaches for formulating the questions of systematic reviews of complex interventions. It also provides conceptual and operational guidance for the use of theory in early stages of systematic reviews of complex interventions.

Paper 3: Adapting frameworks to develop protocols

Butler M, Epstein RA, Totten A, Whitlock EP, Ansari MT, Damschroder LJ, Balk E, Bass EB, Berkman ND, Hempel S, Iyer S, Schoelles K, Guise JM. AHRQ Series on Complex Intervention Systematic Reviews – Paper 3: Adapting frameworks to develop protocols. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.013.

http://www.jclinepi.com/article/S0895-4356(17)30632-7/fulltext

This paper describes and gives examples of adaptations as well as alternatives to traditional analytic frameworks for systematic reviews of complex interventions.

Paper 4: Selecting Analytic Approaches

Viswanathan M, McPheeters ML, Murad MH, Butler ME, Devine EE(B), Dyson MP, Guise J-M, Kahwati LC, Miles JNV, Morton SC. AHRQ Series on Complex Intervention Systematic Reviews – Paper 4: Selecting Analytic Approaches. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.014.

http://www.jclinepi.com/article/S0895-4356(17)30639-X/fulltext

This paper addresses the uncertainty that systematic reviewers face in selecting methods for reviews of complex interventions. Specifically, it lays out parameters for systematic reviewers to consider when selecting analytic approaches that best answer the questions at hand, and suggests analytic techniques that may be appropriate in different circumstances.

Paper 5: Advanced Analytic Methods

Pigott T, Noyes J, Umscheid CA, Myers E, Morton SC, Fu R, Sanders-Schmidler GD, Devine EE(B), Murad MH, Kelly MP, Fonnesbeck C, Petticrew M, Kahwati L, Beretvas SN. AHRQ Series on Complex Intervention Systematic Reviews – Paper 5: Advanced Analytic Methods. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.015.

http://www.jclinepi.com/article/S0895-4356(17)30640-6/fulltext

This paper presents advanced analytic approaches that address four common questions that guide reviews of complex interventions: (1) How effective is the intervention? (2) For whom does the intervention work and in what contexts? (3) What happens when the intervention is implemented? and (4) What decisions are possible given the results of the synthesis?

Paper 6: PRISMA-CI Extension Statement & Checklist

Guise J-M, Butler ME, Chang C, Viswanathan M, Pigott T, Tugwell P, for the Complex Interventions Workgroup. AHRQ Series on Complex Intervention Systematic Reviews – Paper 6: PRISMA-CI Extension Statement & Checklist. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.016.

http://www.jclinepi.com/article/S0895-4356(17)30641-8/fulltext

This paper provides a stand-alone extension to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting tool for complex interventions—PRISMA-CI—to help authors report results clearly and in a way that publishers and readers can understand and use systematic reviews of complex interventions.

Paper 7: PRISMA-CI Elaboration & Explanation

Guise J-M, Butler M, Chang C, Viswanathan M, Pigott T, Tugwell P, for the Complex Interventions Workgroup. AHRQ Series on Complex Intervention Systematic Reviews – Paper 7: PRISMA-CI Elaboration & Explanation. Journal of Clinical Epidemiology (2017). doi: http://dx.doi.org/10.1016/j.jclinepi.2017.06.017.

http://www.jclinepi.com/article/S0895-4356(17)30642-X/fulltext

The Elaboration and Explanation document explains the meaning and rationale for each PRISMA-CI checklist item and provide examples to assist systematic review authors in operationalizing PRISMA-CI guidance.

References

  1. Umscheid C, Whitlock E, Fu R, Berliner E, Paynter R, Anderson J, Motu’apuaka M, Trikalinos T. Systematic Reviews of Complex Multicomponent Health Care Interventions. Research White Paper. AHRQ Publication No. 14-EHC003-EF. Rockville, MD: Agency for Healthcare Research and Quality. March 2014.
  2. McDonald KM, Chang C, Schultz E. Through the Quality Kaleidoscope: Reflections on the Science and Practice of Improving Health Care Quality. Closing the Quality Gap: Revisiting the State of the Science. Methods Research Report. (Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. 290-2007-10062-I.) AHRQ Publication No. 13-EHC041-EF. Rockville, MD: Agency for Healthcare Research and Quality. February 2013.