Skip Navigation
Department of Health and Human Services www.hhs.gov
  • Home
  • Search for Research Summaries, Reviews, and Reports
 
 

EHC Component

  • EPC Project

Full Report

Related Products for this Topic

Save this page in Facebook.com  Save this page in Myspace.com  Save this page in Twitter.com  Save this page on your Google Home Page  Save this page in Windows Live
Save this page in Yahoo  Save this page in Ask.com  Stumble this page.  Save this page in del.ico.us  Digg this page. 

E-mail E-mail   Print Print

Add to My Collections



Research Protocol – Mar. 29, 2011

Comparative Effectiveness of Interventions for Adolescents and Young Adults With Autism Spectrum Disorders

Formats

Table of Contents

Background and Objectives for the Systematic Review

Autism spectrum disorders (ASDs) are among the most common neurodevelopmental disorders, with an estimated prevalence of 1 in 110 children in the United States having an ASD.1 Though these Centers for Disease Control and Prevention data do not include analysis by age group, they suggest that there are a significant number of adolescents and young adults currently affected by an ASD, as well as a significant cohort of children with ASD approaching the adolescent age range. Fombonne estimates that 55,000 individuals between the ages of 15 and 17 in the United States have an ASD.2

Disorders within the autism spectrum include autistic disorder, Asperger syndrome, and pervasive developmental disorder–not otherwise specified. ASDs are characterized by significant impairments in social interaction, behavior, and communication.3 The expression and severity of symptoms of ASD differ widely, and treatments include a range of behavioral, psychosocial, educational, medical, and complementary approaches4-6 that vary by an individual’s age and developmental status.

ASDs are thought to remain lifelong disabilities. Goals of treatment for young children often focus on alleviating core deficits in communication, social interactions, and restricted behaviors. It is hypothesized that ameliorating these fundamental core deficits may help children optimize developmental progress in the short and long terms.7 For many individuals, the core symptoms of ASD (impairments in communication and social interaction and restricted/repetitive behaviors and interests) may see improvements with intervention and over time;8-11 however, deficits typically remain throughout the lifespan although developmental expression may vary. As such, as children transition to adolescence and young adulthood developmentally appropriate interventions to ameliorate core deficits may often continue, but the focus of treatment often shifts toward promoting adaptive behaviors that can facilitate and enhance independent functioning.12 Treatments for some must frequently take into account new emergent symptoms, as well as engagement with new developmental challenges (e.g., independent living, vocational engagement, postsecondary education).

There is also evidence to suggest that specifically the young adult years might be a time of higher risk for individuals with ASDs because improvements in symptoms and in problem behaviors may slow down or stop after youths with ASDs leave high school.13

Current data suggest that attainment of independent living or employment in adulthood for individuals with an ASD is variable, with factors that predict the ability to live and work independently not well elucidated.12 Most individuals with an ASD will require some sort of intervention throughout adolescence and adulthood, and the estimated costs of medical and nonmedical (e.g., special education, daycare) care are prodigiously high. One study estimated the total yearly societal per capita cost of caring for and treating a person with autism in the United States at $3.2 million and at about $35 billion for an entire birth cohort of individuals with autism.14 A study of health care utilization in a large group health plan revealed increased medication costs in older children with an ASD when compared with younger children, as well as similarly aged adolescents without an ASD; other care costs were also higher in this population, including a significantly increased rate of hospitalizations.15

Costs of transitional and employment programs are also high for young adults with an ASD. A recent analysis of U.S. Federal- and State-funded vocational rehabilitation programs showed that the prevalence of ASDs among those in training programs increased from 0.2 percent to 0.6 percent from 2002 to 2006; those with ASDs were among the most costly of nine disability groups examined, with costs even higher among those with ASDs and another comorbid disability. These data also showed, however, that those with ASDs had a higher rate of employment (40.8%) at the time of case closure when compared with those with other disabilities, though with fewer work hours and lower wages than some other disability groups.16

There is no cure for ASDs and currently no global consensus regarding which intervention strategies are most effective. Chronic management, often using multiple treatment approaches, may be required to maximize ultimate functional independence and quality of life by minimizing the core ASD features, facilitating development and learning, promoting socialization, reducing maladaptive behaviors, and educating and supporting families. Investigators in the area have noted that less research on therapies for adolescents or young adults exists than for younger children17 and that such research is increasingly critical as the prevalence of ASDs continues to grow and as children with ASD diagnoses reach adolescence.

The current review grew out of a recognition that care for adolescents and young adults with ASDs varies greatly across care providers and that clinicians and families must make important health care decisions that will have tremendous impact on outcomes with little guidance or knowledge. Similarly, lawmakers struggle with making the best decisions about policy and funding due to lack of an adequate knowledge base regarding the most effective treatments.

The Key Questions

Our draft key questions (KQs; see Appendix A) were posted for public comment on the Agency for Healthcare Research and Quality Effective Health Care Program Web site from December 21, 2010, until January 18, 2011. Commentators typically did not provide their affiliations; however, they included individuals involved in State government, physical therapy, and nursing, as well as parents of individuals with an ASD. Comments on the draft questions addressed interventions, outcomes, and access to care and are summarized below.

Interventions

  • Respondents noted a need to specify speech-language therapy, occupational therapy, independent-living skills training, applied behavior analysis, and residential supports (vs. the more narrow “residential treatment”) in the list of interventions. We will add these approaches and note that the list was not meant to be exhaustive but, rather, illustrative. We will seek studies of any intervention for symptoms or associated features of ASDs in adolescents or young adults. To be sure that we adequately address studies of allied health approaches such as speech/language, occupational, and physical therapies, we will add the Cumulative Index of Nursing and Allied Health Literature (CINAHL) database to the list of databases searched for the review.
  • Respondents noted that categorizing interventions can be an important factor in determining payment issues. We anticipate that we will modify the categorization approach used in a previous review of therapies for children with ASDs as needed to encompass interventions aimed at the adolescent and young adult population.18
  • Respondents commented that determining the “amount” of an intervention required to demonstrate an effect and the relative quality of such effects are of critical importance in making treatment decisions. We will capture and describe information about the intensity and duration of interventions and their effects in the review.

Outcomes

  • Respondents noted that outcome measures are variable across ASD research studies and lack standardization. We will capture and report data on instruments used to assess outcomes and how that may affect interpretation of the literature.
  • Respondents commented on the need to understand long-term effects (no timeframe specified) of interventions and the durability/longevity of effects. We will capture data about length of followup and long-term outcomes as reported in studies meeting our inclusion criteria.
  • Respondents noted that the review should address whether certain interventions work better for certain subgroups of individuals with ASDs (e.g., particular diagnoses, IQ ranges, severity, etc.). We will capture such data about potential modifiers of treatment effectiveness as presented in the literature and stratify our discussion of findings as possible.
  • Respondents noted that motor impairments (e.g., movement dysfunction and impairments, delayed motor skills) are an important area underlying individuals’ functional abilities and that motor outcomes should be addressed. We will capture data regarding such outcomes as they are reported. As noted, we will add the CINAHL database to our list of data sources to help ensure that we capture studies potentially relevant to motor skills indexed in the allied health literature.
  • Respondents commented that our question addressing family-focused interventions (i.e., interventions aimed at improving elements of family functioning such as parental or sibling distress) conflates the impact of ASD on the family and the impact of family-focused interventions. While we acknowledge that ASD undoubtedly has significant effects on family functioning, exploring the effects of ASD itself on the family is beyond the scope of this review. This review will address the comparative effectiveness of interventions for ASDs in adolescents and young adults.

    We will modify our KQ related to family outcomes, however, to clarify our focus on the effect of interventions on the family. We also note that we will capture data related to family characteristics, changes in family functioning, and changes in individuals with ASDs resulting from interventions.

Access to Care

  • Respondents commented that issues with gaining access to treatment, including legal and procedural issues, often complicate care for individuals with ASD. As noted, the focus of this review is on the effectiveness of therapies for ASD in adolescents and young adults. We acknowledge that access to care is critically important; however, investigating such issues is beyond the scope of the current review. We will discuss issues of access to care in the introduction and summary portions of the review.

We also reviewed the KQs and PICOTS with our Technical Expert Panel (TEP). The TEP recommended that we clarify the following areas:

  • Specifying occupational and vocational attainment as an outcome
  • Clarifying that “access to services” includes access to health and other services
  • Adding obesity and attention deficit hyperactivity disorder among our examples of common medical and mental health comorbidities
  • Clarifying the wording of our KQ related to the transition process to state explicitly that it addresses the effectiveness of interventions designed to support the transitioning process
  • Placing the KQ related to harms of interventions after the KQ related to the effectiveness of transitional interventions to clarify that harms may be related to any intervention
  • Noting explicitly that family outcomes include satisfaction related to an intervention
  • Adding the use of public programs as an outcome of interest

We also discussed the 13-to-30 age range with the TEP, noting that we chose the cut-off age of 30 based on advice from our content experts that individuals over the age of 30 typically face different issues related to functioning with an ASD than do individuals under 30 and to ensure that we cast a wide net in identifying studies of potential relevance to young adults.

Modified Key Questions and PICOTS

Our modified KQs and PICOTS are as follows:

Question 1

Among adolescents and young adults with autism spectrum disorders (ASDs), what are the effects of available interventions on the core symptoms of ASD?

Available interventions may include the following broad categories: social skills, functional behavioral interventions, applied behavior analysis, targeted educational interventions, psychoeducational interventions, sexual education, vocational and independent living skills training, transition support, case management, residential supports, psychopharmacology, complementary and alternative medicine, diet/nutrition therapies, crisis management, allied health (e.g., speech/language, physical, and occupational therapies), exercise/recreational interventions, and family-focused interventions.

Question 2

Among adolescents and young adults with ASDs, what are the effects of available interventions on common medical and mental health comorbidities (e.g., epilepsy, sleep disorders, motor impairments, obesity, depression, anxiety, acute and episodic aggression, attention deficit hyperactivity disorder, etc.)?

Question 3

Among adolescents and young adults with ASDs, what are the effects of available interventions on functional behavior, attainment of goals toward independence, educational attainment, occupational/vocational attainment, life satisfaction, access to health and other services, legal outcomes, and social outcomes?

Question 4

Among adolescents and young adults with ASDs, what is the effectiveness of interventions designed to support the transitioning process, specifically to affect attainment of goals toward independence, educational attainment, occupational/vocational attainment, life satisfaction, access to health and other services, legal outcomes, and social outcomes?

Question 5

Among adolescents and young adults with ASDs, what harms are associated with available interventions?

Harms are defined by the Evidence-based Practice Center Program as all possible adverse consequences of an intervention, including adverse events.

Question 6

What are the effects of interventions on family outcomes?

For each KQ, the relevant population, interventions, comparators, outcomes, timing, and setting are as follows:

Population(s):

  • Adolescents and young adults (ages 13–30) with ASDs (autistic disorder, pervasive developmental disorder–not otherwise specified, Asperger syndrome)
  • Parents and family members of adolescents and young adults with ASDs

Interventions:

  • Behavioral interventions including: social skills approaches, cognitive behavioral therapy, functional behavioral interventions, parent training, and applied behavioral analysis-based approaches
  • Educational interventions including: special schools, peer training, sexual education
  • Vocational/occupational- and transition-focused interventions including: vocational support, sheltered workshops, independent-living skills training
  • Medical and other interventions including:
    • U.S. Food and Drug Administration (FDA)-approved medications for the treatment of irritability in ASD: risperidone/Risperdal and aripiprazole/Abilify
    • Serotonin reuptake inhibitors (citalopram/Celexa, escitalopram/Lexapro, fluoxetine/Prozac)
    • Psychostimulants (methylphenidate, amphetamines)
    • Secretin
    • Immunoglobulin
    • Antiviral agents (amantadine/Symadine)
    • Cholinesterase inhibitors (donepezil/Aricept, rivastigmine/Exelon)
    • Pentoxifylline
    • Dimercaptosuccinic acid
    • Dietary supplements (melatonin, polyunsaturated fatty acids, iron, magnesium-vitamin B6)
    • Specialized diets (gluten and casein free diets, restrictive diets)
    • Hyperbaric oxygen chambers
  • Allied health interventions including speech/language, occupational, and physical therapies
  • Complementary and alternative medicine interventions including acupuncture and massage
  • Crisis management
  • Case management
  • Exercise and recreational interventions
  • Family-focused interventions
  • Residential supports

Comparators:

  • Placebo, usual care, or active control

Outcome measures for each question:

At this time, we have identified a set of anticipated outcome areas that we will assess in this review. We will look for each of these broad outcome areas and report on the way they are operationalized by the researchers. In this way, we intend to capture the maximum information available in an anticipated small literature base.

  • Changes in intermediate outcome areas, including social skills/interaction, language and communication, repetitive and other maladaptive behaviors, motor outcomes, psychological distress, adaptive skills development, academic skills development, and family outcomes including family distress and family satisfaction related to interventions. Intermediate outcomes are those that occur directly as a result of the intervention and that may also have longer term implications for the ultimate, functional outcomes that are the long-term goal of therapies.
  • Changes in long-term functional outcome areas, including adaptive independence/self care, academic/occupational/vocational engagement and attainment, psychological well-being, psychosocial adaptation, residential outcomes, legal outcomes, social/relationship-focused outcomes (interpersonal relationships, community involvement/societal participation, self-actualization and acceptance, etc.), access to health services (conservatorship, access to day care, access to health care, access to social, financial, and other support systems), and use of public programs.
  • Adverse effects of intervention(s)
    • Adverse behavioral or psychosocial reactions to behavioral or other therapies (e.g., increased aggression or anxiety)
    • Regression of language, skills, or behaviors
    • Increases in or worsening of comorbid symptoms
    • Adverse reactions to drug therapies (e.g., somnolence, weight gain)
    • Reduction in and negative influences on quality of life

Timing:

  • We will not set specific criteria for timing but will record the timing of interventions as reported in studies meeting our inclusion criteria. We will capture the duration of an intervention (e.g., 6 weeks, 12 months) and the timing of assessments during and after an intervention (e.g., assessments every 4 weeks during treatment administration and at 6 months postintervention).

Settings:

  • Settings include: clinical settings such as a clinic or doctor’s office; the home; residential/vocational treatment settings such as group homes and vocational rehabilitation offices; and educational settings such as high schools, colleges, and special education classrooms.

Analytic Framework

The analytic framework in Figure 1 illustrates the population, interventions, and outcomes that will guide the literature search and synthesis. Individuals with ASD make treatment choices that are informed by each person’s unique socioeconomic and familial context, belief systems, access to care, insurance status, and other factors. Treatment choices can lead to intermediate and long term outcomes including changes in language/communication abilities, adaptive independence, or harms related to treatment. Numbers within the circles on the diagram indicate areas in which key questions to be addressed in the review come into play.

Figure 1: Analytic framework for Comparative Effectiveness of Therapies for Adolescents and Young Adults With ASDs

The analytic framework in Figure 1 illustrates the population, interventions, and outcomes that will guide the literature search and synthesis.  Individuals with autism spectrum disorders between the ages of 13 and 30 and family members of these individuals make treatment choices that are informed by the unique socioeconomic and familial context, belief systems, access to care, insurance status, and other factors. Treatment choices can lead to intermediate or long term outcomes including changes in language/communication abilities, adaptive independence, and harms related to treatment.

Abbreviations: KQ = key question.

Methods

A. Criteria for Inclusion/Exclusion of Studies in the Review

Table 1 lists the inclusion/exclusion criteria we selected based on our understanding of the literature, the topic-refinement phase, input from content experts, and established principles of methodological quality.

As an inclusion criterion, we will set the cut-off level for study size at a minimum of 20 participants with ADS in our target age range. We selected this study size based on a previous review of therapies for children with ASD, which used a sample size of 10 for behavioral, educational, allied health, and complementary and alternative medicine studies and 30 for medical studies.18

Interventions to address ASDs are frequently behavioral in nature and highly intensive. They are also frequently adapted to be targeted to specific study participants given the significant heterogeneity of individuals with ASD. In part because this makes behavioral research quite complex and intensive, study sizes tend to be very small. A cutoff sample size of 20 provides a balance, allowing us to review and comment on adequate literature for the review but with studies large enough to suggest effects of the interventions.

We will limit the review to studies published between 1980 and the present in order to ensure that we cast a broad net related to potential interventions for this age range. Given a lack of translation resources, we will also focus the review on studies published in English; included studies may include non–U.S. populations but must be published in English. In the opinion of the team of clinical experts participating in the review, the majority of ASD research is published in English, regardless of the country of origin or native language of the researchers.

We reviewed these criteria with the TEP who largely concurred with our approach. In particular, the TEP noted that a sample size of 20 for studies of all types of interventions (e.g., medical, behavioral, etc.) takes into account the typically smaller sample sizes in the ASD literature but should still allow us to draw meaningful inferences from the data presented. Some TEP members also questioned the inclusion of case series, noting their generally lower methodological quality and tendency to report only positive results. While we agree that case series without independent replication generally provide limited data, they comprise a significant portion of the ASD literature, as noted in a previous AHRQ review of therapies for ASD in children.18 Large case series may also lend support to smaller controlled studies of interventions for ASD, as well as provide harms data.

We will include case series meeting our inclusion criteria; however, we will not develop evidence tables for or further analyze those case series assessed as poor in quality. Where provided, we will take into account harms data presented in case series in assessing the strength of evidence related to harms.

We also discussed strategies for dealing with papers that include participants with ASD in our target age range as well as those over 30 or under 12 years. We will include papers with a mixed age population provided that 1) the mean age of the participants with ASD is within the 13 to 30 age range or 2) at least 50 percent of the participants with ASD are within the target age range.

We anticipate that some studies may include a mixed disability population as well (i.e., some participants with ASD and some with other disorders). We will extract and report on data related solely to the ASD population where possible. As we proceed with full-text review and data extraction, we will flag studies in which extracting data only on participants with ASD is not possible and determine methods for handling such studies. Methods may include setting a minimum proportion of individuals with ASD that must be included in the study population (e.g., 75% of participants have ASD) or potentially contacting investigators to obtain full data.

Table 1: Inclusion/Exclusion Criteria
Category Criteria
Abbreviations:  ASD = autism spectrum disorder; PDD-NOS = pervasive developmental disorder–not otherwise specified.
Study population

Adolescents or young adults (ages 13–30) with ASD (autism, Asperger syndrome, PDD-NOS) or families/caregivers of individuals with ASD between the ages of 13–30

Time period 1980–present
Publication languages English only
Admissible evidence (study design and other criteria)
Admissible designs

Controlled trials, observational studies including prospective and retrospective cohort studies, prospective and retrospective case series with N ≥ 20 individuals between 13–30 years of age with ASD

Other criteria
  • Original research studies that provide sufficient detail regarding methods and results to enable use and adjustment of the data and results
  • Patient populations must include adolescents or young adults (13–30 years of age) with ASD or families/caregivers of individuals with ASD between the ages of 13–30
  • Studies must address one or more of the following:
    • Treatment modality aimed at modifying ASD core symptoms, common comorbidities, or family-related outcomes or assisting with transitional issues
    • Outcomes (including harms) related to interventions for ASD
  • Studies must include extractable data on relevant outcomes, including data presented in text or tables (vs. solely in figures)

B. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies To Answer the Key Questions

  1. Databases. To ensure comprehensive retrieval of relevant studies of therapies for adolescents and young adults with ASD, we will use four key databases: the MEDLINE medical literature database via the PubMed interface, the PsycINFO psychology and psychiatry database, the Cumulative Index of Nursing and Allied Health Literature (CINAHL), and the Education Resources Information Center database (ERIC). The search strategies for each of these databases will focus specifically on terms related to ASD and treatment or transitional issues, including key words, subject headings, and a combination of subject headings and/or key words (e.g., autism, ASD, therapy, therapeutics, etc.).
  2. Search updates. During our reviews of abstracts and full-text articles, we will update the literature search quarterly by adding relevant studies as needed. We will also update the search when the draft report is submitted and add relevant studies as needed while the draft report is undergoing peer review. We will also incorporate studies that meet our inclusion criteria or are relevant as background material that may be identified by both public and peer reviewers.
  3. Hand searching. We will carry out hand searches of the reference lists of recent systematic reviews or meta-analyses of therapies for ASD; the investigative team will also scan the reference lists of articles that are included after the full-text review phase for studies that potentially could meet our inclusion criteria.
  4. Grey literature. We will request Scientific Information Packets and regulatory information searches addressing those medications with FDA-approval for treating irritability in ASD (risperidone and aripiprazole) as well as on hyperbaric oxygen chambers.

C. Data Abstraction and Data Management

  1. Data-extraction forms. We will develop data-collection forms for the abstract review, the full-text review, and data extraction. The forms used for the abstract review will contain questions about the primary exclusion and inclusion criteria. The forms used for the full-text review are more detailed and are intended to assist in a) identifying studies that meet inclusion criteria and b) initially sorting the studies according to the KQs. Finally, data-extraction forms will collect those data necessary to create evidence tables and perform data synthesis. We anticipate that these data will include those related to baseline participant characteristics (age, diagnosis, symptom severity, etc.), intervention characteristics (description, fidelity/adherence, etc.), and outcomes.

    Prior to data collection, we will develop lists of potential confounders and effect modifiers (e.g., age, IQ, simultaneous therapies/synergistic effects, comorbidities/coexisting conditions, sociocultural context, etc.) and expected outcomes for the data-extraction form that will be informed by our clinical expertise. The form also will include a field in which to report the funding source of a study.

    After reviewing a sample of relevant articles, the Methods and Content Leads will design the data-collection forms and test them on multiple articles before beginning each stage of data extraction. We expect that the data-collection forms will undergo several revisions after these tests are completed.
  2. Initial review of abstracts. We will review all the titles and abstracts identified through our searches against our inclusion/exclusion criteria. Each abstract will be reviewed by at least two members of the investigative team. When differences between the reviewers arise, we will err on the side of inclusion. For studies without adequate information to make the determination, we will retrieve the full-text articles and review them against the inclusion/exclusion criteria.
  3. Retrieving and reviewing articles. We will retrieve and review all articles that meet our predetermined inclusion criteria or for which we have insufficient information to make a decision about eligibility. Each article will be reviewed by at least two members of the investigative team. Differences between the reviewers will be adjudicated by a third party.
  4. Deciding which outcomes are to be extracted. We will identify specific outcome measures to extract in consultation with the TEP and based on our clinical expertise, our initial scan of the literature, and our abstract review. Pre-specifying each measure to extract is not feasible given the large number of assessment tools/measures used in the ASD literature.

    For studies that meet the conditions of the 2nd-round assessment, the abstractors will extract key data and study-quality elements from the article(s) and enter them into evidence tables. We anticipate that these elements will include population and intervention characteristics such as age, diagnoses, intervention approach and dosage; assessment characteristics including instruments used and fidelity measures employed; and outcomes reported. A second reviewer will review those data-extraction forms against the original articles for quality control. Differences in data coding between the abstractor and the reviewer will be resolved by consensus.

    We will develop a simple categorization scheme for coding the reasons that articles, at the stage of full review, are not finally included in the report. The abstractor will note the reason(s) for exclusion on the article abstraction form. We will then record those codes in an EndNote® (Thomson Reuters, New York, NY) bibliographic database so that we can later compile a listing of excluded articles and the reasons for such exclusions.

D. Assessment of Methodological Quality of Individual Studies

We will assess study quality by using a modified version of the quality-assessment approach developed in a previous review on therapies for children with ASD18 and informed by the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.19 This quality-assessment approach considered factors related to study design, diagnostic approach, participant ascertainment, intervention characteristics, outcomes measurement, and statistical approach and included questions such as: Did the study employ a group design (employ a comparison group)? Did the authors report differences in or hold steady all concomitant interventions? Were outcomes coded and assessed by individuals blinded to the intervention status of the participants? And for randomized controlled trials, was there an intent-to-treat analysis?

As we proceed with full-text review and data extraction, we will determine relevant modifications to the approach to ensure that we address critical aspects of the literature on young adults and adolescents with ASD. Two senior investigators will independently assess each included study with disagreements between assessors resolved through discussion to reach consensus.

E. Data Synthesis

  1. Preparing evidence tables. We will enter data into evidence tables by using predetermined abbreviations and acronyms consistently across all entries. The dimensions (i.e., areas of special focus, or the columns) of each evidence table may vary by KQ as appropriate, but the tables will contain some common elements, such as author, year of publication, study location (e.g., country, city, state) and time period, population description, sample size, and study type (e.g., randomized controlled trial, prospective observational study, etc.).
  2. Synthesizing results. Given significant differences in populations, interventions, and outcomes measured in the ASD literature, we anticipate largely qualitative synthesis of findings. Within each KQ, we will organize results by study design, with a focus on those designs less subject to bias (i.e., randomized controlled trials, controlled trials), those studies rated as higher quality in our quality assessment process, and those employing comparison groups.

F. Grading the Evidence for Each Key Question

Assessing the strength of evidence. We will also utilize explicit criteria for rating the overall strength of the collective evidence on each KQ into qualitative categories (e.g., low, moderate, high, insufficient). We will use established concepts of the quantity of evidence (e.g., numbers of studies, aggregate ending-sample sizes), the quality of evidence (from the quality ratings on individual articles), and the coherence or consistency of findings across similar and dissimilar studies and in comparison to known or theoretically sound ideas of clinical or behavioral knowledge. We will make these judgments as appropriate for each of the KQs.

The strength of evidence evaluation will be that stipulated in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews,19 which emphasizes the following four major domains: risk of bias (low, medium, high), consistency (inconsistency not present, inconsistency present, unknown or not applicable), directness (direct, indirect), and precision (precise, imprecise). Risk of bias is derived from the quality assessment of the individual studies that addressed the KQ and specific outcome under consideration. Each key outcome on each comparison of interest will be given an overall evidence grade based on the ratings for the individual domains.

The overall strength of evidence will be graded as “high” (indicating high confidence that the evidence reflects the true effect and further research is very unlikely to change our confidence in the estimate of effect), “moderate” (indicating moderate confidence that the evidence reflects the true effect and further research may change our confidence in the estimate of effect and may change the estimate), “low” (indicating low confidence that the evidence reflects the true effect and further research is likely to change our confidence in the estimate of effect and is likely to change the estimate), or “insufficient” (indicating that evidence is either unavailable or does not permit estimation of an effect). When no studies are available for an outcome or comparison of interest, we will grade the evidence as insufficient.

Two senior staff will independently grade the body of evidence; disagreements will be resolved as needed through discussion or third-party adjudication. We will record strength of evidence assessments in tables, summarizing for each outcome.

G. Assessing Applicability

We will assess the applicability of findings reported in the included literature to the general population of adolescents and young adults with ASDs by determining the population, intervention, comparator, and setting (PICOS) in each study and developing an overview of these elements for each intervention category. We will also review potential modifiers of effect of treatment to identify subgroups, which may include different age groups, specific ASD diagnoses, or level of education.

We anticipate that participants in trials of antipsychotics and other medications are likely clinic-based and more selected than the general population. We also anticipate variation in the scope of transitional and other services offered across the country.

References

  1. Autism and Developmental Disabilities Monitoring Network Surveillance 2006 Principal Investigators for the Centers for Disease Control and Prevention. Prevalence of autism spectrum disorders—Autism and Developmental Disabilities Monitoring Network, United States, 2006. MMWR Surveill Summ 2009;58(SS-10):1-20.
  2. Fombonne E. Epidemiological surveys of autism and other pervasive developmental disorders: an update. J Autism Dev Disord 2003;33:365-82.
  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington, DC: American Psychiatric Publishing; 2000.
  4. Seida JK, Ospina MB, Karkhaneh M, et al. Systematic reviews of psychosocial interventions for autism: an umbrella review. Dev Med Child Neurol 2009;51:95-104.
  5. Parr J. Autism. Clin Evid (Online) 2008;2008. PII: 0322.
  6. Ospina MB, Krebs Seida J, Clark B, et al. Behavioural and developmental interventions for autism spectrum disorder: a clinical systematic review. PLoS One 2008;3:e3755.
  7. Myers SM, Johnson CP. Management of children with autism spectrum disorders. Pediatrics 2007;120:1162-82.
  8. Shattuck PT, Seltzer MM, Greenberg JS, et al. Change in autism symptoms and maladaptive behaviors in adolescents and adults with an autism spectrum disorder. J Autism Dev Disord 2007;37:1735-47.
  9. McGovern CW, Sigman M. Continuity and change from early childhood to adolescence in autism. J Child Psychol Psychiatry 2005;46:401-8.
  10. Fecteau S, Mottron L, Berthiaume C, et al. Developmental changes of autistic symptoms. Autism 2003;7:255-68.
  11. Seltzer MM, Krauss MW, Shattuck PT, et al. The symptoms of autism spectrum disorders in adolescence and adulthood. J Autism Dev Disord 2003;33:565-81.
  12. Seltzer MM, Shattuck P, Abbeduto L, et al. Trajectory of development in adolescents and adults with autism. Ment Retard Dev Disabil Res Rev 2004;10:234-47.
  13. Taylor JL, Seltzer MM. Changes in the autism behavioral phenotype during the transition to adulthood. J Autism Dev Disord 2010;40:1431-46.
  14. Ganz ML. The lifetime distribution of the incremental societal costs of autism. Arch Pediatr Adolesc Med 2007;161:343-9.
  15. Croen LA, Najjar DV, Ray GT, et al. A comparison of health care utilization and costs of children with and without autism spectrum disorders in a large group-model health plan. Pediatrics 2006;118:e1203-11.
  16. Cimera RE, Cowan RJ. The costs of services and employment outcomes achieved by adults with autism in the US. Autism 2009;13:285-302.
  17. Schall C, McDonough J. Autism spectrum disorders in adolescence and early adulthood: characteristics and issues. J Vocational Rehabil 2010;32:81-8.
  18. Agency for Healthcare Research and Quality. Evidence-based Practice Center systematic review protocol. Therapies for Children with Autism Spectrum Disorders. Available at: http://effectivehealthcare.ahrq.gov/ehc/products/106/366/Therapies%20for%20Autism%20Spectrum%20Disorder%20%282-9-2010%29.pdf. Accessed March 28, 2011.
  19. Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—Agency for Healthcare Research and Quality and the Effective Health-Care Program. J Clin Epidemiol 2010;63:513-23.
  20. Chou R, Aronson N, Atkins D, et al. AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care Program. J Clin Epidemiol 2010;63:502-12.

Definition of Terms

  • We define medical and related interventions as those in which a foreign substance is administered to the body.
  • Harms are defined by the Evidence-based Practice Center Program as the totality of all possible adverse consequences of an intervention.20
  • Applied behavior analysis is an umbrella term describing principles and techniques used in the assessment, treatment, and prevention of challenging behaviors and the promotion of new desired behaviors. The goal of applied behavioral analysis is to teach new skills, promote generalization of these skills, and reduce challenging behaviors with systematic reinforcement.
  • Crisis management describes techniques and strategies to handle significant behavioral or other problems or issues arising for an individual with an ASD.
  • Case management refers to the coordination of care and services for an individual with an ASD.
  • Exercise and recreational interventions are those interventions employing physical exercise or recreational activities such as hiking or swimming.
  • Family-focused interventions are interventions aimed at improving the functioning of the family of an individual with an ASD. Functioning may include family or sibling distress.
  • Residential supports includes any intervention dealing with housing-related issues for an individual with ASD. Supports include group homes as well as functional supports within the home to aid an individual with an ASD with activities of daily living (e.g., reminder cards, schedules, etc.).
  • The Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition,3 defines the essential diagnostic features of disorders on the ASD spectrum as follows:
    • Autism: “…[T]he presence of markedly abnormal or impaired development in social interaction and communication and a markedly restricted repertoire of activity and interests. Manifestations of the disorder vary greatly depending on the developmental level and chronological age of the individual. Autistic Disorder is sometimes referred to as early infantile autism, childhood autism, or Kanner's autism.”
    • Asperger Syndrome: “…[S]evere and sustained impairment in social interaction (Criterion A) and the development of restricted, repetitive patterns of behavior, interests, and activities (Criterion B). The disturbance must cause clinically significant impairment in social, occupational, or other important areas of functioning (Criterion C). In contrast to Autistic Disorder, there are no clinically significant delays or deviance in language acquisition (e.g., single non-echoed words are used communicatively by age 2 years, and spontaneous communicative phrases are used by age 3 years) (Criterion D), although more subtle aspects of social communication (e.g., typical give-and-take in conversation) may be affected. In addition, during the first 3 years of life, there are no clinically significant delays in cognitive development as manifested by expressing normal curiosity about the environment or in the acquisition of age-appropriate learning skills and adaptive behaviors (other than in social interaction) (Criterion E). Finally, the criteria are not met for another specific Pervasive Developmental Disorder or for Schizophrenia (Criterion F)….”
    • Pervasive Developmental Disorder–Not Otherwise Specified: “…Severe and pervasive impairment in the development of reciprocal social interaction associated with impairment in either verbal or nonverbal communication skills or with the presence of stereotyped behavior, interests, and activities, but the criteria are not met for a specific Pervasive Developmental Disorder, Schizophrenia, Schizotypal Personality Disorder, or Avoidant Personality Disorder.”

Summary of Protocol Amendments

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.

  1. Review of Key Questions
    For all EPC reviews, key questions were reviewed and refined as needed by the EPC with input from Key Informants and the Technical Expert Panel (TEP) to assure that the questions are specific and explicit about what information is being reviewed. In addition, for Comparative Effectiveness reviews, the key questions were posted for public comment and finalized by the EPC after review of the comments.
  2. Key Informants
    Key Informants are the end users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions. Within the EPC program, the Key Informant role is to provide input into identifying the Key Questions for research that will inform healthcare decisions. The EPC solicits input from Key Informants when developing questions for systematic review or when identifying high priority research gaps and needed new research. Key Informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

    Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their role as end-users, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
  3. Technical Experts
    Technical Experts comprise a multi-disciplinary group of clinical, content, and methodologic experts who provide input in defining populations, interventions, comparisons, or outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism.

    Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
  4. Peer Reviewers
    Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for CERs and Technical briefs, be published three months after the publication of the Evidence report.

    Potential Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

Appendix A: Draft Key Questions

Question 1

Among adolescents and young adults with autism spectrum disorders (ASDs), what are the effects of available interventions on the core symptoms of ASD?

Available interventions will include the following broad categories: social skills, psychopharmacology, functional behavioral interventions, psychoeducational interventions, vocational training, targeted educational interventions, transition support, complementary and alternative medicine (CAM), diet/nutrition, crisis management, sexual education, case management, family-focused interventions, recreational interventions, and residential treatment.

Question 2

Among adolescents and young adults with ASDs, what are the effects of available interventions (see KQ1) on common medical and mental health comorbidities (e.g., epilepsy, sleep disorders, depression, anxiety, acute and episodic aggression, etc.)?

Question 3

Among adolescents and young adults with ASDs, what are the effects of available interventions (see KQ1) on functional behavior, attainment of goals toward independence, educational attainment, occupational attainment, life satisfaction, residential outcomes, social outcomes, and relationship-focused outcomes?

Question 4

Among adolescents and young adults with ASDs, what harms are associated with available interventions (see KQ1)?

Harms are defined by the Evidence-based Practice Center Program as the totality of all possible adverse consequences of an intervention.

Question 5

Among adolescents and young adults with ASDs, what is the effectiveness of available interventions (see KQ1) to assist with common transitional issues, specifically to affect attainment of goals toward independence, educational attainment, occupational attainment, life satisfaction, access to services, legal outcomes, and social outcomes?

Question 6

What are the effects on family adaptation and family outcomes of interventions directed either toward adolescents and young adults with ASDs or toward their families?

Return to Top of Page