Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care

Article Alert

The free Article Alert service delivers a weekly email to your inbox containing the most recently published articles on all aspects of systematic review and comparative effectiveness review methodologies.

  • Medical, psychological, educational, etc., methodology research literatures covered
  • Curated by our seasoned research staff from a wide array of sources: PubMed, journal table of contents, author alerts, bibliographies, and prominent international methodology and grey literature Web sites
  • Averages 20 citations/week (pertinent citations screened from more than 1,500 citations weekly)
  • Saves you time AND keeps you up to date on the latest research

Article Alert records include:

  • Citation information/abstract
  • Links: PMID (PubMed ID) and DOI (Digital Object Identifier)
  • Free Full Text: PubMed Central or publisher link (when available)
  • RIS file to upload all citations to EndNote, RefWorks, Zotero, or other citation software

To sign up for free email updates of Article Alert, contact the Scientific Resource Center at


The Article Alert for the week of June 6, 2016 (sample articles)

Fearns N, Kelly J, Callaghan M, Graham K, Loudon K, Harbour R, Santesso N, McFarlane E, Thornton J, Treweek S. What do patients and the public know about clinical practice guidelines and what do they want from them? A qualitative study. BMC Health Serv.Res. 2016 Feb 24;16(1):74. PMID: 27121606.

Background: Guideline producers are increasingly producing versions of guidelines for the public. The aim of this study was to explore what patients and the public understand about the purpose and production of clinical guidelines, and what they want from clinical guidelines to support their healthcare decisions.
Methods: Participants were purposively selected to represent a range of the likely users of patient versions of guidelines, including individuals with health conditions (diabetes and depression), general members of the public, health communication professionals and a group of young people. Participants were asked about their awareness and understanding of clinical guidelines and presented with scenario recommendations, or draft materials from patient guidelines to prompt discussion. Each discussion was facilitated by one or two researchers. All focus groups were recorded and transcribed prior to analysis. Data were analysed using framework analysis.
Results: We ran nine focus groups involving 62 individuals, supplemented by four interviews with people experiencing homelessness. Eight groups were held in Scotland, one in England. The four interviews were held in Scotland. The framework analysis yielded five themes: access and awareness; what patients want to know; properties of guidelines; presenting evidence; and format. Awareness of guidelines was low. Participants emphasised the need for information that enables them to choose between treatment options, including harms. They would like help with this from healthcare professionals, especially general practitioners. Participants differed in their support for the inclusion of numerical information and graphs.
Conclusions: Members of the public want information to help them choose between treatments, including information on harms, particularly to support shared decisions with health professionals. Presenting numerical information is a challenge and layered approaches that present information in stages may be helpful. Ignoring the themes identified in this study is likely to lead to materials that fail to support public and patient healthcare decision making.


Kelly SE, Moher D, Clifford TJ. Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines. Syst.Rev. 2016 May 10;5(1):79. PMID: 27160255.

Background: Rapid reviews are an accelerated evidence synthesis approach intended to meet the timely needs of decision-makers in healthcare settings. Quality of conduct and reporting has been described in the rapid review literature; however, no formal assessment has been carried out using available instruments. The objective of this study was to explore compliance with conduct and reporting guidelines in rapid reviews published or posted online during 2013 and 2014.
Methods: We performed a comprehensive literature search for rapid reviews using multiple bibliographic databases (e.g. PubMed, MEDLINE, EMBASE, the Cochrane Library) through December 31, 2014. Grey literature was searched thoroughly, and health technology assessment agencies were surveyed to identify additional rapid review products. Candidate reviews were assessed for inclusion using pre-specified eligibility criteria. Detailed data was collected from the included reviews on study and reporting characteristics and variables significant to rapid reviews (e.g. nomenclature, definition). We evaluated the quality of conduct and reporting of included rapid reviews using the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklists. Compliance with each checklist item was examined, and the sum of adequately reported items was used to describe overall compliance. Rapid reviews were stratified to explore differences in compliance related to publication status. The association between compliance and time to completion or length of publication was explored through univariate regression.
Results: Sixty-six rapid reviews were included. There were heterogeneous nomenclature, research questions and approaches to rapid reviews. Compliance with AMSTAR and PRISMA checklists was poor. Published rapid reviews were compliant with individual PRISMA items more often than unpublished reviews, but no difference was seen in AMSTAR item compliance overall. There was evidence of an association between length of publication and time to completion and the number of adequately reported PRISMA or AMSTAR items.
Conclusions: Transparency and inadequate reporting are significant limitations of rapid reviews. Scientific editors, authors and producing agencies should ensure that the reporting of conduct and findings is accurate and complete. Further research may be warranted to explore reporting and conduct guidelines specific to rapid reviews and how these guidelines may be applied across the spectrum of rapid review approaches.


Pfadenhauer L, Rohwer A, Burns J, Booth A, Lysdahl K B, Hofmann B, Gerhardus A, Mozygemba K, Tummers M, Wahlster P, et al. Guidance for the Assessment of Context and Implementation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions: The Context and Implementation of Complex Interventions (CICI) Framework. January 2 2016.

[Description from the guidance document]

Challenges in assessments of health technologies

In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies which:

  • Are complex, i.e. consist of several interacting components, target different groups or organizational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring
  • Are context-dependent - current HTA usually focusses on the technology, not on the system within which it is used
  • Perform differently depending on the way they are implemented
  • Have different effects on different individuals

Furthermore, HTA usually assesses and appraises aspects side-by-side, while decision-making needs an integrated perspective on the value of a technology. In the EU-funded INTEGRATE-HTA project, we developed concepts and methods to deal with these challenges, which are described in six guidances.

Where context-dependency and implementation of a technology matter, it is critical to make this explicit, to document available information as fully as possible and to assess if and how this may affect intervention reach and effectiveness.

Purpose and Scope of the Guidance: The purpose of this guidance is to provide a framework to commissioners, producers and users of systematic reviews and health technology assessments (HTA) that allows for the systematic conceptualisation, assessment and documentation of the setting, context and implementation of a complex intervention. It presents an overarching framework comprising eight domains of context (i.e. setting, geographical, epidemiological, socio-cultural, socio-economic, ethical, legal and political issues) and four domains of implementation (i.e. provider, organisation and structure, funding and policy), including definitions and descriptions of each of these domains.

Development of the Guidance: The Context and Implementation of Complex Interventions (CICI) framework was developed in an iterative fashion. Based on a scoping review of existing theories, models and frameworks concerned with "context", "setting" and "implementation", an initial framework for conceptualising, assessing and documenting the interacting dimensions of context (including setting) and implementation was developed. Due to the lack of conceptual clarity identified in the scoping review, and in order to provide a sound scientific basis for this framework, systematic literature searches were performed for the concepts "setting", "context" and "implementation". The three concepts were appraised as partially mature. Pragmatic utility concept analysis, as developed by Morse and colleagues, was selected as the most suitable method to evaluate concept use by comparing and contrasting applications across the health field. During data extraction, relevant constructs informing the selected theories, models and frameworks were also extracted and integrated with the initial CICI framework. This revised initial framework was applied in four systematic reviews (three quantitative and one qualitative review), as well as in one HTA. The findings from the applications were used to create the revised CICI framework presented in this guidance.

Application of the Guidance: The CICI framework can be applied in HTA and systematic reviews of effectiveness, as well as in qualitative systematic reviews. The guidance provides definitions and descriptions of domains of context and implementation and provides examples which may be of relevance for each domain. In addition, it proposes a list of questions to assess each domain: a) to retrieve quantitative information about the domain (which characteristics influence …?) and b) to generate a more in-depth understanding of the domain's influence (how do the characteristics influence …?). Moreover, the list encourages the researcher to assess relevant interactions between domains (e.g. ethical and socio-cultural domain). Additionally, we suggest a graphical representation of the domains contained in the CICI framework that supports researchers in systematically assessing domains of context and implementation.

The CICI framework can moreover be used to assess the applicability of a technology to a specific context. The domains serve as the basis for a semi-structured questionnaire that can be used with experts when exploring potential contextual barriers and facilitators to the implementation of a specific technology.

Conclusion: The CICI framework can be applied in quantitative, qualitative or mixed-method systematic reviews and in HTAs. In quantitative systematic reviews, it supports the documentation of relevant context and implementation aspects through data extraction, examination of heterogeneity and subgroup analysis. In qualitative or mixed-method systematic reviews, it serves as a starting point for identifying and formulating specific research questions to understand how context and/or implementation influence the intervention. In HTAs, an appraisal of the respective domains can be conducted by asking stakeholder panels about their importance and relevance. Overall, the framework helps to present context and implementation issues in an integrated fashion, and supports the assessments of applicability and generalizability of HTAs and systematic reviews.