Background and Objectives for the Systematic Review
Asthma is a chronic inflammatory disorder of the airways, characterized by varying degrees of airflow obstruction. Bronchoconstriction, inflammatory cell infiltration, and airway edema reduce airflow intermittently, often in response to specific exposures, resulting in respiratory symptoms.1 In the United States (U.S.), the current prevalence of asthma has increased over the past decade, from an estimated 22.2 million Americans in 2005 to 24.0 million Americans in 2014.2,3 Asthma can significantly impact patients' and families' quality-of-life and ability to pursue activities such as school, work, and exercise. Globally, asthma ranks 14th based on the burden of disease, as measured by disability adjusted life years.4 In the U.S., asthma contributes significantly to healthcare resource utilization and associated costs. For example, in 2012, asthma was one of the top twenty leading diagnosis groups for primary care visits and was the main reason for 1.8 million emergency department visits and 439,000 hospitalizations. While the severity of disease varies between patients and over time in the same patient, asthma can be fatal, accounting for approximately 1 death per 100,000 Americans.5
Effectiveness of Indoor Inhalant Allergen Reduction
Control of environmental factors that may contribute to asthma is one of the four components of asthma management. Many common indoor inhalant allergens have been associated with increased risk of asthma exacerbations, including pollen, animal dander, house dust mites (HDM), mice, cockroaches, mold, and others.6 Numerous interventions have been designed to reduce exposure to allergens in the environment where asthma patients live, work, learn, play, and sleep.7 Examples of these interventions include carpet removal, use of specially designed mattress covers and pillowcases, air filtration systems, pest-elimination techniques, and containment or removal of family pets.
Evaluating the effectiveness of allergen exposure reduction interventions presents multiple challenges. Strategies to control environmental factors often include multipronged approaches, resulting in difficulty identifying the effectiveness of individual interventions. Similarly, some interventions are designed to reduce or eradicate exposure to multiple allergens simultaneously, and the evidence addressing individual allergens varies. Another challenge is that research may be limited by inadequate or inconsistent measures of allergen exposure reduction.
It is also important to consider that implementing environmental control strategies may require substantial financial resources or structural modifications to the home environment. Some interventions may therefore not be feasible for patients who live in poverty or for tenants who lack authority to modify a dwelling.
Role of Bronchial Thermoplasty in the Management of Asthma
Patients with severe and persistent asthma are managed with multiple medications that may include inhaled, orally administered, and biologic therapeutics. For these patients, bronchial thermoplasty (BT) is another treatment option, requiring multiple procedures performed by a physician. The procedure involves visualization of the airway using bronchoscopy and then using a probe (catheter) to deliver heat to the airway wall. The thermal energy is intended to debulk, or reduce excess smooth muscle in the treated airways.8 In April 2010, the United States Food and Drug Administration (FDA) approved the Alair BT system for use in patients 18 years of age and older with severe, persistent asthma.
Purpose of the Systematic Review
In 1989, the National Heart, Lung, and Blood Institutes (NHLBI) initiated the National Asthma Education and Prevention Program (NAEPP) to address growing concern about asthma in the U.S. One of the first accomplishments of the NAEPP was to convene a panel of experts who produced a report, National Asthma Education and Prevention Program Expert Panel Report (EPR): Guidelines for the Diagnosis and Management of Asthma, in 1991. The guidelines address the diagnosis, evaluation, and treatment of asthma. Given the most recent report, EPR-3, was published in 2007,1 NHLBI assessed the need for an update by requesting information from the public, NAEPP Coordinating Committee Members and its affiliates, and members of the 2007 Expert Panel. Collected information was provided to the NHLBI Advisory Council Asthma Expert Working Group, which produced a report to summarize the process and recommendations from their needs assessment.9 The Working Group identified six high priority topics that should be updated. For each topic, key questions meriting a systematic literature review were formulated. NHLBI engaged AHRQ to perform the systematic reviews through its Evidence-based Practice Centers (EPC). This document represents the systematic review of "The Effectiveness of Indoor Allergen Reduction and the Role of Bronchial Thermoplasty in the Management of Asthma." The review will also highlight areas of controversy and identify needs for future research on these priority areas.
Key Question 1
Among individuals with asthma, what is the effectiveness of interventions to reduce or remove exposure to indoor inhalant allergens on asthma control, exacerbations, quality of life, and other relevant outcomes?
- What is the effectiveness of carpet removal or cleaning?
- What is the effectiveness of covers for pillows, mattresses, or furniture; and laundering of linens?
- What is the effectiveness of pest elimination, pet removal, and pet bathing?
- What is the effectiveness of air purifiers, mold removal, moisture reduction, ventilation, and insulation?
- What is the effectiveness of multicomponent interventions that include more than one strategy and/or affect more than one allergen?
Key Question 2
What are benefits and harms of using bronchial thermoplasty in the treatment of adult (≥18 years) patients with severe asthma in addition to standard treatment?
PICOTS (Population, Intervention, Comparator, Outcome, Timing, Setting)
- Key Question (KQ) 1
- Specify how asthma was diagnosed, when this information is available
- Allergen specific IgE sensitization, skin testing results, eosinophils, if this information is available
- All severity (National Heart, Lung, and Blood Institute [NHLBI]) treatment steps 1 through 6)
- All ages
- KQ 2
- Specify how asthma was diagnosed, when this information is available
- NHLBI steps 4-6
- Age ≥18 years
- KQ 1
- Reduction or elimination of exposure to indoor inhalant allergen(s)
- Cats, dogs
- Rodents, pests
- House dust mites
- Wall-to-wall versus area rugs
- Cleaning (professional services; high efficiency particulate air filtration (HEPA) vacuums)
- Linens and furniture
- Pillow/mattress covers
- Furniture covers/"wipe-down" furniture
- Frequent laundering of linens
- Animals and insects
- Pet bathing
- Pet removal or restriction of pet access
- Pest control (professional and lay interventions)
- Air quality
- Air purifiers
- Mold removal
- Ventilation or duct cleaning
- Removal of clutter
- Multicomponent interventions
- Multiple strategies implemented concurrently
- Interventions that affect multiple allergens
- Indoor aeroallergens
- Reduction or elimination of exposure to indoor inhalant allergen(s)
- KQ 2
- Bronchial thermoplasty
- KQ 1
- No intervention to reduce or eliminate exposure to indoor inhalant allergen(s)
- Reduction or elimination of exposure to different indoor inhalant allergen(s)
- Reduction or elimination of exposure to multiple indoor inhalant allergens
- KQ 2
- Treatments used in patients with severe asthma excluding thermoplasty
Outcomes, all KQs
- Asthma control
- Composite measures*
- Asthma Control Test (ACT) / Childhood ACT
- Asthma Control Questionnaire (ACQ)
- Other composite measures as reported by studies
- Individual measures
- Nighttime waking
- Symptom free days
- Maximum symptom days
- Airway hyperresponsiveness
- Growth indicators (pediatric patients)
- Composite measures*
- Systemic corticosteroids for asthma
- Asthma-specific hospitalizations
- Asthma-specific ED visits
- Asthma-specific urgent care visits (other than ED)
- Asthma-specific admissions to intensive care unit, or intubations
- Quality of life*
- Asthma Quality of Life Questionnaire (AQLQ)
- Pediatric Asthma Quality of Life Questionnaire (PAQLQ)
- Pediatric Asthma Caregivers Asthma Quality of Life Questionnaire (PACQLQ)
- Asthma-specific days missed from work or school
- Participation in sports and recreational activities
- Death, asthma-specific and all cause
- Health care utilization and costs
- Asthma-specific ambulatory care visits
- Asthma-specific medication use (including medication name, dose, duration)
- Hospitalizations, ED visits, urgent care visits
- All cause
- Associated with potentially asthma-related complications
- Myocardial infarction
- Steroid-induced hypoglycemia
- Additional Outcomes for KQ 1
- Indoor inhalant allergen levels measured by formal testing
- Psychological burden/emotional distress
- Pet removal
- Limited ability to implement interventions for patients who rent and/or who live in poverty
- Financial burden as reported in the studies
- Patients and families
- Purchase of pillow and mattress covers, air purifiers, pest-elimination strategies, insulation, mold-removal products, and other interventions
- Carpet removal, purchase of special vacuums, professional cleaning
- Health care providers
- Time/personnel costs associated with education, home visits, eradication of allergens, as reported in the studies
- Patients and families
- Additional Outcomes for KQ 2
- Adverse events associated with BT*
- Airway irritation (cough, wheezing, dyspnea, chest discomfort)
- Airway compromise
- Upper or lower respiratory tract infections
- Lung collapse
- Adverse events associated with BT*
Any duration of followup
- KQ 1
- KQ 2
- Clinical settings
*Proposed critical outcomes for grading strength of evidence
Figure 1. Analytic framework for Non-pharmacologic Management of Asthma
KQ: Key Question
Criteria for Study Inclusion and Exclusion
As suggested in the Agency for Healthcare Research and Quality (AHRQ) EPC Methods Guide for Comparative Effectiveness Reviews, the inclusion criteria are listed below in separate categories pertaining to publication type, study design, patient characteristics, test characteristics, and reported data.10
- Full-length articles. The article must be published as a full-length, peer-reviewed study. Abstracts and meeting presentations will not be included because they do not include sufficient details about experimental methods to permit an evaluation of study design and conduct; they may also contain only a subset of measured outcomes.11,12 Additionally, it is not uncommon for abstracts that are published as part of conference proceedings to have inconsistencies when compared with the final study publication or to describe studies that are never published as full articles.13-17
- Redundancy. To avoid double-counting patients, when several reports of the same or overlapping groups of patients are available, only outcome data from the report with the largest number of patients will be included. We will make an exception and include data from a smaller study when it reports data on an outcome that was not provided by the largest report or reports longer followup data for an outcome.
- English language. When a study with an English abstract is published in a foreign language, the abstract will be assessed against the full set of inclusion/exclusion criteria. If the study appears to fit the inclusion criteria, then we will evaluate whether excluding the study may result in language bias (e.g., if the findings differ from other included studies.) If language bias seems unlikely, the study will be excluded. If a study is selected for inclusion, it will be translated and the data extracted into the evidence tables.
Study Design Criteria
Systematic reviews, meta-analyses, and randomized controlled trials (RCTs) will be considered for inclusion. Non-randomized interventional studies with concurrent controls (e.g., non-randomized trials) or historical controls (e.g., pre-post studies) will also be considered for inclusion. Case reports that describe adverse events associated with bronchial thermoplasty will be considered for inclusion for KQ2. In vivo, in vitro, and animal studies will be excluded. Systematic reviews will be screened to identify additional studies that may meet our inclusion criteria, and to provide relevant context in the Discussion section of the final report. A systematic review may also be used as a primary source of data if 3 conditions are met: 1) the review is determined to be at low risk of bias (using Cochrane's ROBIS tool); 2) the included studies would individually meet our inclusion criteria; and 3) our searches did not identify additional, relevant, primary studies that meet our criteria.
- For KQ 1, studies must compare an exposure control intervention for a specific allergen(s) to one of the following: no intervention; a different exposure control intervention for the same allergen(s); any exposure control intervention for a different allergen(s); or multicomponent interventions.
- For KQ 2, comparative studies must compare BT to other interventions for controlling severe adult asthma. Case series and case reports will be considered for inclusion if they describe adverse events associated with thermoplasty that have not been described in the comparative studies.
To be included, a study must have reported data obtained from patients who have received a diagnosis of asthma of any severity (KQ 1) or severe asthma (KQ 2). Patients receiving step 4, 5 or 6 treatment will be defined as having severe asthma. Studies that include patients with allergic rhinitis or other pulmonary disorders, in addition to patients with asthma, will be included if either: a) outcomes data were stratified by asthma diagnosis; or b) at least 85 percent of the study population had a diagnosis of asthma. If data were reported separately for patients with asthma, then all analyses will use the asthma specific data.
For KQ 2, patients must be aged 18 years or older. Studies will be included if at least 85 percent of patients were adults, or if outcomes data are reported separately for patients aged 18 or older.
- For KQ 1, a specific intervention(s) for a specific allergen(s) must be examined. Allergens include household pets, rodents, pests, cockroaches, dust mites, and mold.
Interventions include carpet removal or cleaning; use of covers for pillows, mattresses, or furniture; frequent laundering of linens; pest elimination; pet removal or frequent bathing; mold removal; and use of air purifiers, moisture reduction, ventilation, and/or insulation.
Interventions must be compared to other interventions for the same or a different allergen(s), or to multicomponent interventions for one or more allergens, or to no intervention for allergen exposure control. For carpet-related interventions, studies must compare carpet removal to: no removal; carpet removal in specific rooms (e.g., bedrooms); enhanced carpet cleaning (e.g., using vacuums with HEPA filtration); or removal of wall-to-wall carpeting versus use of area rugs.
- For KQ 23, BT must be examined. Studies that compare thermoplasty to any pharmacologic or non-pharmacologic treatment for severe asthma will be included.
- The study must report data pertaining to one of the outcomes of interest (see the Key Questions section for a list).
- We will include data from time points and outcomes reported from studies with at least 20 patients (or 10 patients per study group) with asthma who represent at least 50 percent of the patients originally enrolled in the study.
Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions
Literature searches will be performed by Medical Librarians at the Evidence-based Practice Center (EPC) Information Center, and will follow established systematic review protocols. We will search the following databases using controlled vocabulary and text words: EMBASE, MEDLINE, PubMed, and The Cochrane Library.
The following gray literature sources will be searched using text words: ClinicalTrials.gov, Centers for Disease Control and Prevention (CDC), Medscape, National Academy of Medicine, National Guideline Clearinghouse™ (NGC), the United States Environmental Protection Agency (EPA), the United States Food and Drug Administration (FDA), the United States National Institute of Environmental Health Sciences, and the Web sites of relevant organizations (e.g., Agency for Healthcare Research and Quality [AHRQ], American Academy of Allergy, Asthma & Immunology [AAAAI], American Academy of Pediatrics [AAP], American College of Allergy, Asthma, and Immunology [ACAAI], American Lung Association [ALA], American Public Health Association [APHA], American Thoracic Society [ATS]), Asthma and Allergy Foundation of America [AAFA], Children's Health Protection Advisory Committee [CHPAC], Global Initiative for Asthma [GINA], National Center for Healthy Housing [NCHH], National Environmental Education Foundation [NEEF], and National Heart, Lung, and Blood Institute [NHLBI]. An example search strategy is shown in the Appendix. Scientific Information Packets (SIPs) submitted by interested parties will also be reviewed.
Literature screening will be performed in duplicate using the database Distiller SR (Evidence Partners, Ottawa, Ontario, Canada). Literature search results will initially be screened for relevancy. Relevant abstracts will be screened against the inclusion and exclusion criteria in duplicate. Studies that appear to meet the inclusion criteria will be retrieved in full and screened again in duplicate against the inclusion and exclusion criteria. All disagreements will be resolved by consensus discussion between the two original screeners. The literature searches will be updated during the Peer Review process, before finalization of the review.
Data Abstraction and Data Management
Data will be abstracted using Microsoft Word and Excel. All data will be checked for accuracy by a second reviewer. Elements to be abstracted include: general study characteristics (e.g., study design, objective, setting, enrolled number of patients, length of follow-up and inclusion of seasonal variation); patient characteristics (e.g., age, sex, ethnicity, homeowner or renter, inner-city versus suburban/rural, asthma severity, allergen specific sensitization, comorbidities, exposure to second-hand smoke); allergen(s) targeted by the intervention; details of interventions; outcomes data; and risk of bias items.
Assessment of Methodological Risk of Bias of Individual Studies
We will use the Cochrane Collaboration's tool for assessing risk of bias in RCTs. Studies will be rated as "Low," "High," or "Unclear" risk of bias. For non-randomized studies, we will use Cochrane's instrument for evaluating Risk of Bias in Non-randomized Studies – of Interventions (ROBINS-I). Studies will be rated as "Low," "Moderate," "Serious," "Critical," or "No information." Risk of bias will be assessed by two independent reviewers, and discrepancies will be addressed through consensus discussion. We will contact authors of original studies if we determine that additional information is needed. If we are unable to reach the study authors or receive a response within 6 weeks, we will assess the study without additional input.
For studies reporting on patient-oriented clinical outcomes, we plan to perform meta-analysis when appropriate and possible. Decisions about whether meta-analysis is appropriate will depend on the judged clinical homogeneity of the different study populations, research designs, and outcomes. When meta-analysis is impossible (due to limitations of reported data) or is judged to be inappropriate, the data will be synthesized using a descriptive, narrative review.
We will compute effect sizes and measures of variance using standard methods and will perform random-effects meta-analysis using the Hartung-Knapp method.18,19 Meta-regression and subgroup analysis will be used to explore possible causes of heterogeneity. Potential covariates include population descriptors (e.g., age, baseline asthma severity, comorbidities, family income/socioeconomic factors), settings, and types of allergens and interventions.
Subgroup analyses will be performed to isolate effects potentially associated with specific populations. Subgroups for KQ 1 will be identified according to the following criteria: type of allergen, patient age (pediatric versus adult), asthma severity, setting (e.g., home, school, work, daycare), and, for home-based studies, type of ownership (homeowner versus renter.)
Critical outcomes for all Key Questions are expected to include asthma-control composite measures, asthma-exacerbation measures, and asthma-related quality of life, as described in the PICOTS framework. For KQ 2, adverse events will also be considered critical outcomes. Input from the clinical investigators, Technical Expert Panel (TEP), peer reviewers, and sponsoring agency will also be considered in the identification and final selection of critical outcomes.
Grading the Strength of Evidence (SOE) for Major Comparisons and Outcomes
We will use a formal grading system that conforms with the EPC Methods Guide recommendations on grading the SOE.10,20 The primary domains assessed in this system include risk of bias, directness, consistency, precision, and publication bias. Additional domains may be used when appropriate, including dose-response association, strength of association, and the possibility that all plausible confounders would increase the effect size. The output is a rating of the SOE: high, moderate, low, or insufficient. This rating is made separately for each outcome of each comparison of each KQ.
If the evidence is sufficient to permit a conclusion, the rating is deemed high, moderate, or low. A rating of insufficient will be given when the evidence does not permit a conclusion for the outcome of interest for that KQ. Below, we discuss the primary domains and how they will be considered as inputs to the ratings:
Risk of bias (see the section Assessment of Methodological Risk of Bias of Individual Studies above). Study limitations concern the degree to which the included studies for a given outcome have a high likelihood of adequate protection against bias (i.e., have good internal validity). If the evidence permits a conclusion, then, all else being equal, a set of studies at low risk of bias yields a higher SOE rating than a set of studies at moderate or high risk of bias.
Directness. Directness relates to (a) whether evidence links interventions directly to a health outcome of specific importance for the review, and (b) for comparative studies, whether the comparisons are based on head-to-head studies.
Consistency. Consistency is the degree to which included studies find either the same direction or similar magnitude of effect.
Precision. Precision is the degree of certainty surrounding an effect estimate with respect to a given outcome, based on the sufficiency of sample size, number of events, and width of confidence intervals relative to a clinically important effect estimate.
Reporting bias. Reporting bias will be addressed by examining the funding source of included studies, the direction and magnitude of effects identified in included studies, and noting the presence of abstracts or ClinicalTrials.gov entries describing studies that did not subsequently appear as full-length published articles. For interventions with at least 10 studies that present data for critical outcomes, review of funnel plots may be used to help ascertain publication bias.
Several a priori factors may limit the applicability of findings. Small sample size may be an important limitation in many studies, and addressing this through meta-analysis may be challenging if there is substantial heterogeneity in study design, intervention, and outcome reporting. Additionally, many studies may focus on children who are at high risk of developing asthma and may not represent the general asthma population. Similarly, the populations included in existing studies are likely to underrepresent racial minorities, patients with substantial comorbidities (such as diabetes), and obese patients, although these groups are at higher risk for poor asthma-related health outcomes. Exposure to second-hand smoke is also an important factor that affects pulmonary function and asthma-related outcomes, but which may not be adequately represented or documented in clinical studies. A final consideration is that many patients with asthma in the real world may be renters rather than homeowners and may, therefore, have limited opportunities to implement certain types of interventions, even if they prove useful in controlled studies.
- Expert Panel Report 3: guidelines for the diagnosis and management of asthma. National Asthma Education and Prevention Program, Third Expert Panel on the Diagnosis and Management of Asthma. Bethesda (MD). Report No: 07-4051. Washington, DC: National Heart, Lung, and Blood Institute; 2007.
- Most recent asthma data. [internet]. Atlanta, GA: Centers for Disease Control and Prevention (CDC); [accessed 2016 Jul 05]. Available: http:www.cdc.gov/asthma/most_recent_data.htm.
- Trends in asthma morbidity and mortality. [internet]. Chicago, IL: American Lung Association; [accessed 2016 Apr 01]. Available: http://www.lung.org/assets/documents/research/asthma-trend-report.pdf.
- The global asthma report 2014. [internet]. Auckland, New Zealand: Global Asthma Network; [accessed 2016 Jul 19]. Available: http://www.globalasthmareport.org/resources/Global_Asthma_Report_2014.pdf.
- Asthma national health statistics. [internet]. Atlanta, GA: Centers for Disease Control and Prevention (CDC); [accessed 2016 Apr 01]. Available: http://www.cdc.gov/nchs/fastats/asthma.htm.
- Asthma overview. [internet]. Milwaukee (WI): American Academy of Allergy Asthma & Immunology; [accessed 2016 Apr 07]. [2 p]. Available: http://www.aaaai.org/conditions-and-treatments/asthma.
- Gotzsche PC, Johansen HK. House dust mite control measures for asthma. Cochrane Database Syst Rev. 2008;(2):CD001187. Also available: http://dx.doi.org/10.1002/14651858.CD001187.pub3. PMID: 18425868.
- ECRI Institute. Bronchial thermoplasty (Alair System) for treating adult patients with severe symptomatic asthma. Plymouth Meeting (PA): ECRI Institute; 2014 Oct. 34 p. (Emerging Technology Evidence Report).
- Draft needs assessment report for potential update of the Expert Panel Report-3 (2007): Guidelines for the Diagnosis and Management of Asthma. [internet]. January 2015. National Heart, Lung, and Blood Advisory Council Asthma Expert Working Group; [accessed 2016 Jul 19]. Available: http://www.nhlbi.nih.gov/sites/www.nhlbi.nih.gov/files/NHLBAC_Asthma-WG-Report-2015%5B1%5D.pdf.
- Owens D, Lohr K, Atkins D, Treadwell J, Reston J, Bass E, Chang S, Helfand M. Grading the strength of a body of evidence when comparing medical interventions. In: Agency for Healthcare Research and Quality (AHRQ). Methods guide for comparative effectiveness reviews. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2009 Jul. Also available: /sites/default/files/pdf/methods-guidance-grading-strength_methods.pdf.
- Chalmers I, Adams M, Dickersin K, Hetherington J, Tarnow-Mordi W, Meinert C, Tonascia S, Chalmers TC. A cohort study of summary reports of controlled trials. JAMA. 1990 Mar 9;263(10):1401-5. PMID: 2304219.
- Neinstein LS. A review of Society for Adolescent Medicine abstracts and Journal of Adolescent Health Care articles. J Adolesc Health Care. 1987 Mar;8(2):198-203. PMID: 3818406.
- Dundar Y, Dodd S, Williamson P, Dickson R, Walley T. Case study of the comparison of data from conference abstracts and full-text articles in health technology assessment of rapidly evolving technologies: does it make a difference? Int J Technol Assess Health Care. 2006 Jul;22(3):288-94.
- De Bellefeuille C, Morrison CA, Tannock IF. The fate of abstracts submitted to a cancer meeting: factors which influence presentation and subsequent publication. Ann Oncol. 1992 Mar;3(3):187-91. PMID: 1586615.
- Scherer RW, Langenberg P. Full publication of results initially presented in abstracts. In: Cochrane Library [Cochrane methodology review]. Issue 2. Oxford: Update Software; 2001 [accessed 2001 Apr 23]. [35 p]. Available: http://www.cochrane.org/index.htm.
- Yentis SM, Campbell FA, Lerman J. Publication of abstracts presented at anaesthesia meetings. Can J Anaesth. 1993 Jul;40(7):632-4. PMID: 8403137.
- Marx WF, Cloft HJ, Do HM, Kallmes DF. The fate of neuroradiologic abstracts presented at national meetings in 1993: rate of subsequent publication in peer-reviewed, indexed journals. AJNR Am J Neuroradiol. 1999 Jun-Jul;20(6):1173-7. PMID: 10445467.
- Hartung J, Knapp G. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med. 2001 Dec 30;20(24):3875-89. PMID: 21640820.
- Cornell JE, Mulrow CD, Localio R, Stack CB, Meibohm AR, Guallar E, Goodman SN. Random-effects meta-analysis of inconsistent effects: a time for change. Ann Intern Med. 2014 Feb 18;160(4):267-71. Also available: http://dx.doi.org/10.7326/M13-2886. PMID: 24727843.
- Agency for Healthcare Research and Quality (AHRQ). Methods guide for medical test reviews [AHRQ Publication No. 12-EHC017]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2012 Jun. 188 p. Also available: /sites/default/files/pdf/methods-guidance-tests_overview-2012.pdf.
Definition of Terms
Summary of Protocol Amendments
If we need to amend this protocol, we will give the date of each amendment, describe the change, and give the rationale in this section. Changes will not be incorporated into the protocol.
Review of Key Questions
AHRQ posted the key questions on the Effective Health Care Web site for public comment. The EPC refined and finalized the key questions after review of the public comments and input from Key Informants and the TEP. This input is intended to ensure that the key questions are specific and relevant.
Key Informants are the end users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions. Within the EPC program, the Key Informant role is to provide input into identifying the key questions for research that will inform health care decisions. The EPC solicits input from Key Informants when developing questions for systematic review or when identifying high-priority research gaps and needed new research. Key Informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.
Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their role as end-users, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The Task Order Officer (TOO) and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
Technical Experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore, study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor do they contribute to writing the report. They have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.
Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparation of the final report. Peer reviewers do not participate in writing or editing the final report or other products. The final report does not necessarily represent the views of individual reviewers. The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published three months after the publication of the evidence report.
Potential Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.
EPC Team Disclosures
EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.
Role of the Funder
This project was funded under Contract No. xxx-xxx from AHRQ, U.S. Department of Health and Human Services. The TOO reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by AHRQ or the U.S. Department of Health and Human Services.
Appendix. Sample of Search Strategy
|Set Number||Concept||Search Statement|
|1||Asthma||asthma/exp OR 'allergic asthma'/exp OR 'asthmatic state'/exp OR 'extrinsic asthma'/exp OR 'intrinsic asthma'/exp OR 'mild intermittent asthma'/exp OR 'mild persistent asthma'/exp OR 'nocturnal asthma'/exp OR 'occupational asthma'/exp OR 'severe persistent asthma'/exp OR asthma*:ti,ab,de|
|2||Household Allergens||(allergen/exp OR 'environmental exposure'/exp OR 'health hazard'/exp OR 'disease exacerbation'/exp OR allerg* OR irritant* OR trigger* OR exacerbate*) AND ('airborne particle'/exp OR cat/exp OR cockroach/exp OR dander/exp OR dog/exp OR dust/exp OR household/exp OR mite/exp OR mould/exp OR 'pest insect'/exp OR 'pest organism'/exp OR 'pest rodent'/exp OR 'pet animal'/exp OR cat OR cats OR cockroach* OR housedust* OR roach* OR damp* OR dander OR dermatophagoide* OR daycare OR dog OR dogs OR dust* OR home* OR house* OR indoor* OR insect* OR mite OR mites OR mold OR mould OR moldy OR mouldy OR mouse OR mice OR pet OR pets OR pest OR pests OR rodent* OR school* OR moist* OR fungus OR fungi OR chalk*)|
|3||Environmental Interventions||('air filter'/exp OR bed/exp OR cleaning/exp OR 'environmental sanitation'/exp OR vacuum/exp OR 'pests and pest control'/exp OR 'pest control'/exp OR ‘indoor residual spraying’/exp) OR (air NEAR/2 (clean* OR filter* OR filtrat* OR purif*)) OR ventilat* OR insulat* OR (duct* NEAR/2 clean*) OR dehumid* OR bed OR beds OR bedding OR futon* OR clean* OR comforter* OR cover OR covers OR covering* OR duvet* OR encase* OR feather* OR linen* OR fabric OR pillow* OR mattress* OR sanita* OR sanitis* OR sanitiz* OR sheet* OR vacuum* OR sun OR sunlight* OR hypoallergenic OR remove OR removal OR bath* OR exterminat* OR spray* OR ((allergen OR pet OR pets OR pest*) NEAR/5 (reduc* OR avoid* OR eliminat*)) OR wipe OR wiping OR launder OR laundering OR laundry OR hepa OR 'high-efficiency particulate arrestance' OR 'high efficiency particulate arrestance' OR wash OR washing|
|4||Carpet/Flooring Removal||building/exp OR (carpet* OR floor* OR rug OR rugs OR wood*):ab,ti,de|
|5||Combine sets||1 AND 2 AND 3|
|6||Combine sets||1 AND 4|
|7||Bronchial Thermoplasty||Alair* OR (bronch* AND thermoplast*):ab,ti,de OR 'bronchial thermoplasty device'/exp|
|8||Combine sets||5 OR 6|
|9||Remove unwanted publication types (Key Question 1)||8 NOT (abstract:nc OR annual:nc OR book/de OR 'case report'/de OR 'case study'/de OR conference:nc OR 'conference abstract':it OR 'conference paper'/de OR 'conference paper':it OR 'conference proceeding':pt OR 'conference review':it OR congress:nc OR editorial/de OR editorial:it OR erratum/de OR letter:it OR note/de OR note:it OR meeting:nc OR sessions:nc OR 'short survey'/de OR symposium:nc)|
|10||Controlled study filter (Key Question 1)||9 AND ('randomized controlled trial'/exp OR 'randomized controlled trial' OR 'randomization'/exp OR 'randomization' OR 'double blind procedure'/exp OR 'double blind procedure' OR 'single blind procedure'/exp OR 'single blind procedure' OR 'placebo'/exp OR 'placebo' OR 'latin square design'/exp OR 'latin square design' OR 'crossover procedure'/exp OR 'crossover procedure' OR 'triple blind procedure'/exp OR 'triple blind procedure' OR 'controlled study'/exp OR 'controlled study' OR 'clinical trial'/exp OR 'clinical trial' OR 'comparative study'/exp OR 'comparative study' OR 'cohort analysis'/exp OR 'cohort analysis' OR 'follow up'/exp OR 'follow up' OR 'intermethod comparison'/exp OR 'intermethod comparison' OR 'parallel design'/exp OR 'parallel design' OR 'control group'/exp OR 'control group' OR 'prospective study'/exp OR 'prospective study' OR 'retrospective study'/exp OR 'retrospective study' OR 'case control study'/exp OR 'case control study' OR 'major clinical study'/exp OR 'major clinical study' OR 'evaluation study'/exp OR 'evaluation study' OR random*:de OR random*:ti OR placebo* OR (singl* OR doubl* OR tripl* OR trebl* AND (dummy OR 'blind'/exp OR blind OR sham)) OR 'latin square' OR isrctn* OR actrn* OR (nct* NOT nct))|
|11||Systematic Review/Meta-analysis filter||9 AND ('research synthesis' OR pooled OR 'systematic review'/exp OR 'systematic review' OR 'meta analysis'/exp OR 'meta analysis' OR (('evidence base' OR 'evidence based'/exp OR 'evidence based' OR methodol* OR systematic OR quantitative* OR studies OR search*) AND ('review'/exp OR 'review' OR 'review'/it)))|
|12||Combine Sets||10 OR 11|
|13||Remove unwanted publication types (Key question 2)||7 NOT (abstract:nc OR annual:nc OR book/de OR conference:nc OR 'conference abstract':it OR 'conference paper'/de OR 'conference paper':it OR 'conference proceeding':pt OR 'conference review':it OR congress:nc OR editorial/de OR editorial:it OR erratum/de OR letter:it OR note/de OR note:it OR meeting:nc OR sessions:nc OR 'short survey'/de OR symposium:nc)|
|14||RCT or controlled study filter plus case reports (Key Question 2)||13 AND ('randomized controlled trial'/exp OR 'randomized controlled trial' OR 'randomization'/exp OR 'randomization' OR 'double blind procedure'/exp OR 'double blind procedure' OR 'single blind procedure'/exp OR 'single blind procedure' OR 'placebo'/exp OR 'placebo' OR 'latin square design'/exp OR 'latin square design' OR 'crossover procedure'/exp OR 'crossover procedure' OR 'triple blind procedure'/exp OR 'triple blind procedure' OR 'controlled study'/exp OR 'controlled study' OR 'clinical trial'/exp OR 'clinical trial' OR 'comparative study'/exp OR 'comparative study' OR 'cohort analysis'/exp OR 'cohort analysis' OR 'follow up'/exp OR 'follow up' OR 'intermethod comparison'/exp OR 'intermethod comparison' OR 'parallel design'/exp OR 'parallel design' OR 'control group'/exp OR 'control group' OR 'prospective study'/exp OR 'prospective study' OR 'retrospective study'/exp OR 'retrospective study' OR 'case control study'/exp OR 'case control study' OR 'major clinical study'/exp OR 'major clinical study' OR 'evaluation study'/exp OR 'evaluation study' OR random*:de OR random*:ti OR placebo* OR (singl* OR doubl* OR tripl* OR trebl* AND (dummy OR 'blind'/exp OR blind OR sham)) OR 'latin square' OR isrctn* OR actrn* OR (nct* NOT nct)) OR case*:ti OR 'case report'/de OR 'case study'/de|
|15||Systematic Review/Meta-analysis filter||13 AND ('research synthesis' OR pooled OR 'systematic review'/exp OR 'systematic review' OR 'meta analysis'/exp OR 'meta analysis' OR (('evidence base' OR 'evidence based'/exp OR 'evidence based' OR methodol* OR systematic OR quantitative* OR studies OR search*) AND ('review'/exp OR 'review' OR 'review'/it)))|
|16||Combine Sets||14 OR 15|
|17||Combine ALL sets||12 OR 16|
|18||Apply Limits||17 AND ([article in press]/lim OR [humans]/lim OR [in process]/lim)|
- * = truncation character (wildcard)
- NEAR/n = search terms within a specified number (n) of words from each other in any order
- NEXT/n = search terms within a specified number (n) of words from each other in the order specified
- / = search as a subject heading
- exp = "explodes" controlled vocabulary term (e.g., expands search to all more specific related terms in the vocabulary's hierarchy)
- mj = denotes a term that has been searched as a major subject heading
- :de = search in the descriptors field (controlled terms and keywords)
- :lnk = floating subheading
- :it,pt. = source item or publication type
- :ti. = limit to title
- :ti,ab. = limit to title and abstract fields