Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care
  • Home
  • Search for Research Summaries, Reviews, and Reports
 
 

Research Report - Final – Aug. 1, 2011

Frameworks for Determining Research Gaps During Systematic Reviews

Formats

People using assistive technology may not be able to fully access information in this file. For additional assistance, please contact us.

Structured Abstract

Research Objective

Systematic reviews, in addition to summarizing the evidence, generally also discuss needs for future research. However, in contrast to the methods of the systematic review, future needs are not identified systematically. There is limited literature describing organizing principles or frameworks for determining research gaps. We developed and pilot-tested a framework for the identification of research gaps from systematic reviews.

Study Design

We reviewed the research gaps identification practices of organizations involved with evidence synthesis. We contacted: (i) evidence-based practice centers (EPCs) (n=12) associated with the Agency for Healthcare Research and Quality (AHRQ) in the US and Canada, and (ii) other organizations around the world (n=64) that conduct systematic reviews, cost-effectiveness analyses, or technology assessments. Based on the responses, we developed a framework for identifying research gaps. We obtained feedback from two technical experts at our institution and pilot-tested this framework on two randomly selected EPC evidence reports. We also developed a simple, user-friendly worksheet with instructions to facilitate the use of the framework by investigators during or after a systematic review.

Population Studied

Not Applicable.

Principal Findings

Four (33.3%) EPCs and 3 (8.1%) of the other organizations reported currently using an explicit framework to determine research gaps. We did not identify one framework that captured all elements needed to determine and characterize research gaps. Variations of the PICO (population, intervention, comparison, outcomes) framework were most common. It is also important to classify the reason(s) for the gap to help determine how to address the gap. Therefore, we propose a framework that includes both the characterization of the gap using PICOS elements (also including setting) and the identification of the reason(s) why the gap exists. The framework allows investigators to classify reasons for the existence of a research gap as: (a) insufficient or imprecise information, (b) biased information; (c) inconsistency or unknown consistency, and (d) not the right information. We mapped each of these reasons to concepts from three commonly used evidence grading systems: the Grading of Recommendations Assessment, Development and Evaluation (GRADE); the United States Preventive Services Task Force (USPSTF); and the Strength of Evidence (SOE) used by EPCs. This allows leveraging of work already being completed during evidence grading. During pilot-testing, we identified challenges including difficulty in applying the framework for completed systematic reviews and differences in the specificity of research gaps abstracted by different users. These could be tackled with a priori discussions amongst investigators. Further testing should determine if these challenges are ameliorated if the framework is used during a systematic review.

Conclusions

We developed a framework to identify and characterize research gaps from systematic reviews. The framework provides for the classification of where and why the current evidence falls short.

Implications for Policy, Delivery, or Practice

In synthesizing evidence, systematic reviews inform health-care decisions for patients, policymakers, and clinicians. Systematic reviews can also be invaluable for identifying research gaps, thus helping develop research agendas. This potential impact of systematic reviews has not been realized. Our framework provides for systematically identifying and characterizing research gaps from systematic reviews. This explicit identification of research gaps will help determine the type of research needed to address the goals of comparative effectiveness research.

Journal Publications

Robinson KA, Saldanha IJ, McKoy NA. Development of a framework to identify research gaps from systematic reviews. J Clin Epidemiol. 2011 Dec;64(12):1325-30. Epub 2011 Sep 19. PubMed PMID: 21937195.