The Agency for Healthcare Research and Quality (AHRQ) has reported on healthcare quality and disparities since 2003. The National Healthcare Quality and Disparities Report (NHQDR) presents trends for measures related to access to care, affordable care, care coordination, effective treatment, healthy living, patient safety, and person-centered care.¹ The report provides users with the latest available findings on care quality and access to healthcare stratified by diseases and conditions, as well as disparities related to race and ethnicity, income, health insurance status, age, gender, education, setting of care, and type of care. The report is based on more than 250 measures of quality and disparities, based on dozens of data sources, covering a broad array of healthcare services and settings. Settings of care covered include ambulatory care, health centers, emergency departments, hospitals, nursing homes, hospices, and home health.

The current NHQDR Framework is, despite yearly reviews and updates, conceptually based on the 2010 National Academies of Sciences, Engineering, and Medicine (NASEM, formerly IOM) report, Future Directions for the National Healthcare Quality and Disparities Report.1 More recently, the NHQDR refined the framework using the National Quality Strategy.^2,3

The objective of the technical brief is to support an update the 2010 NASEM framework to better align with current focus and priority areas. There are a range of considerations related to selecting a framework for quality of care, in particular when the framework and associated measures of quality of care need to be suitable to detect and document disparities. Updating the framework requires a thorough review of the literature and existing evidence, in particular literature that has been published since 2010. Furthermore, AHRQ has identified new priority areas¹ of health and access to care, i.e., patient safety, care coordination, effective treatment, healthy living, and affordable care. Since the last update, there are also emergent paradigm changes that should be recognized. A key debate in the field in the last decade centers around equity, social determinants of health, and priority populations. In the 2010 NHQDR framework, equity is highlighted as a crosscutting dimension and serves as criterion in the process for ranking measures as well as a distinct data element for inclusion in the national healthcare reports. Extensive research has been conducted regarding operationalizing and measuring equity in recent years.⁴

Another key issue for quality of care is related to the measure criteria of susceptibility to being influenced by the health system. There has been extensive research on how social determinants of health impact healthcare quality and contribute to disparities in health outcomes. The Centers for Disease Control and Prevention (CDC) defines health disparities as preventable differences in the burden of disease, injury, violence, or in opportunities to achieve optimal health experienced by socially disadvantaged racial, ethnic, and other population groups, and communities.^5,6 The CDC defines social determinants of health as conditions in the places where people live, learn, work, and play that affect a wide range of health and quality-of life-risks and outcomes. This suggests that social determinants of health influences health, but it also encompasses various non-medical factors that influence health outcomes, including housing, education, employment, and neighborhood conditions. Disadvantaged communities often experience inadequate housing, environmental pollution, limited educational opportunities, and unsafe neighborhoods, which can negatively impact health. Considering that the influence could be outside the health system, the current criteria “susceptibility to being influenced by the health system” may need to be revisited.

Updating the framework for AHRQ’s NHQDR is a very challenging undertaking. Any chosen framework will determine to a large extent which quality measures are prioritized, which in turn determines which area will be the focus of quality improvement interventions trying to improve quality of care and reduce disparities.

The brief will be facilitated by guiding questions (GQs), documenting research and Key Informant input:

GQ1. Which frameworks have been developed or are used for quality of care?

• What settings, populations, and intended use were the frameworks developed for?
• How are the framework domains defined?
• In what context have these frameworks been used?
• How do these frameworks intersect with levers and tools available to federal and state governments?
• How are the frameworks and domains similar or different from the 2010 NASEM framework?

GQ2. How should the 2010 NASEM framework and its domains be updated?

• How would existing AHRQ NHQDR measures be reorganized in the updated framework and domains?
• Are there available measures for new framework domains?
  • Describe measures in terms of their definition, population, years available, geographic representation, data sources, and supporting evidence.

The methods for this technical brief will follow the Methods Guide for Evidence-based Practice Center (EPC) Program. This technical brief aims to answer the guiding questions with information from interviews with key informants, grey literature searches, and published literature.

1. Data Collection

A. Discussions with Key Informants

We will hold key informant calls to discuss key aspects of the brief’s topic. We selected five areas of interest (with the option of expanding to further content areas) for which we identified representatives: framework development, care models, equity, healthcare improvement, and social determinants of health. For this technical brief, we will use the key informants as a source of information not yet captured in the scientific literature. We will seek the help of the key informants in understanding the conceptual complexity of quality of care and disparities in healthcare delivery systems as well as practical support by asking for input to refine the guiding questions and search strategy. The planned questions are documented in Table 1.

Table 1. Key Informant Questions

The key informant interviews will follow a semi-structured format. Key informant interviews will be conducted as web conferences, and we will invite key informants to individual interviews as their schedule allows. Interviews will be documented during each call. Notes will be reviewed and discussed by the investigators to evaluate how the input provides insight on the key questions. The interviews will be supported by an online survey that informs participants of our questions in advance and that will also give participants the opportunity to add information which may have been lost during the call. The online survey will allow key informants to have the opportunity to express their individual preferences and feel heard. We will also provide the key informants with this technical brief protocol to enable meaningful input and discussions during the call.

B. Gray Literature Search

We will search the websites of health services research organizations, funders of research, federal agencies charged with improving quality of care or those that address health disparities. Specifically, we will review the website of the Robert Wood Johnson Foundation (improving health and wellbeing) and that of the John Hartford Foundation (healthcare practice innovation).

In addition, we will review the websites of CMS (Centers of Medicare and Medicaid Services), HHS (Department of Health and Human Services), VHA (Veterans Health Administration), ASPE (Assistant Secretary for Planning and Evaluation), NIA (National Institute of Aging), NIMHD (National Institute of Minority Health and Health Disparities), and PCORI (Patient-Centered Outcomes Research Institute).

We will also search ESHPI (AcademyHealth’s Evidence-Informed State health Policy Institute) and NAM (National Academy of Medicine) for frameworks and criteria to select quality measures.

C. Published Literature Search

We will search the following literature sources. For primary research studies we will search PubMed (biomedical literature), CINAHL (allied nursing), and PsycINFO (psychosocial literature). Preliminary searches showed the challenge of balancing the search yield and not missing relevant studies and the considerations have informed the draft search strategy for the database PubMed is shown in the appendix. The terminology for quality indicators is not standardized and nomenclature varies across clinical fields. The literature searches use a set of general quality of care indicator terms ("Quality Indicators, Health Care"[Mesh]) combined with search terms for frameworks (e.g., framework, logic model, conceptual model). Searches will use controlled vocabulary where applicable as well as text words as not to miss newer studies not indexed yet.

We will reference-mine existing reviews where available.⁷ The reviews will also be used to continuously refine the search strategy where necessary, in particular to locate grey literature. We will search the same databases used for primary research to identify potentially relevant reviews. In addition, we will search the Cochrane Database of Systematic Reviews and PROSPERO to systematically identify existing research syntheses. The draft search strategy for research databases is documented in the appendix. The search strategy will undergo peer review to ensure all relevant resources for this technical brief have been identified.

Table 2 below describes the eligibility criteria in a Population, Concept, Context, Other limiters framework.

Table 2. Criteria for Inclusion/Exclusion of Studies in the Review

Literature searches targeted to each guiding question will be designed, executed, and documented by the EPC Medical Librarian. Searches will be conducted without date restriction.

2. Data Organization and Presentation

A. Information Management

Literature screening and data abstraction will be conducted in an online database designed for systematic reviews (DistillerSR). Literature reviewers will screen citations supported by machine learning. All citations that at least one reviewer determines to be potentially relevant to the technical brief will be obtained as full text. Full text studies will be screened by two independent reviewers against the explicit eligibility criteria; disagreements will be resolved by consensus. The literature searches will be updated during the peer review and public comment process before finalization of the technical brief. Any identified data meeting the eligibility criteria will be incorporated into the final technical brief.

The data abstraction will capture detailed information about the identified research to facilitate decisions about framework components or conceptualization changes. We will capture the scope and target of the framework, the included domains and components, and we will document the conceptualization (if any) of equity of care or any considerations regarding disparities.

Critical appraisal will focus on the source (e.g., published by an individual author group or endorsement by a professional organization), stakeholder involvement (in the development of the framework), evidence based (components based on a systematic literature review or empirical data), defined population (framework target reported), and validity tested (e.g., goodness of fit assessed, applied in different contexts) following a critical appraisal form developed for a prior framework review.⁸

B. Data Presentation

We will provide KIs and partners with a list of identified frameworks, together with all associated publications, and a list of excluded publications before submitting the draft report of the technical brief.

The synthesis for the technical brief will include summary tables and figures to clearly convey the identified information. An evidence table resulting from the data abstraction will document all identified frameworks clearly to facilitate answers to the guiding questions. We will compare and contrast identified models with the 2010 NASEM framework. The synthesis will be structured by key characteristics, such as the included domains and the role of equity in the model. Where publications depict a conceptual framework, analytic framework, or theoretical model visually, we will seek permission to include the model in an appendix as a resource. We will display figures created under creative commons agreements with appropriate attribution and ask publishers for permission to use the figure.

In addition, we will provide a structured gap analysis that uses the analytic framework and the eligibility criteria dimensions to make detailed recommendations to guide future research.

1. Agency for Healthcare Research and Quality. 2022 National Healthcare Quality and Disparities Reports. Rockville, MD; 2022. Accessed on September 6, 2023.
2. Agency for Healthcare Research and Quality. National Quality Strategy. 2017. Accessed on September 6, 2023.
3. Centers for Medicare & Medicaid Services (CMS). CMS National Quality Strategy.  April 2023.
4. Agniel D, Martino SC, Burkhart Q, et al. Incentivizing Excellent Care to At-Risk Groups with a Health Equity Summary Score. J Gen Intern Med. 2021 Jul;36(7):1847-57. doi: 10.1007/s11606-019-05473-x. PMID: 31713030.
5. Centers for Medicare & Medicaid Services (CMS). Social Determinants of Health at CDC. 2022. Accessed on September 6, 2023.
6. Centers for Medicare & Medicaid Services (CMS). Health Disparities. 2023. Accessed on September 6, 2023.
7. Kanaoka K, Iwanaga Y, Tsujimoto Y, et al. Quality indicators for acute cardiovascular diseases: a scoping review. BMC Health Serv Res. 2022 Jul 5;22(1):862. doi: 10.1186/s12913-022-08239-0. PMID: 35790957.
8. Ferguson L, Gruskin S, Bolshakova M, et al. Frameworks and measures for HIV-related internalized stigma, stigma and discrimination in healthcare and in laws and policies: a systematic review. J Int AIDS Soc. 2022 Jul;25 Suppl 1(Suppl 1):e25915. doi: 10.1002/jia2.25915. PMID: 35818866.

There are no amendments.

Within the Technical Brief process, Key Informants serve as a resource to offer insight into the clinical context of the technology/intervention, how it works, how it is currently used or might be used, and which features may be important from a patient of policy standpoint. They may include clinical experts, patients, manufacturers, researchers, payers, or other perspectives, depending on the technology/intervention in question. Differing viewpoints are expected, and all statements are crosschecked against available literature and statements from other Key Informants. Information gained from Key Informant interviews is identified as such in the report. Key Informants do not do analysis of any kind nor contribute to the writing of the report and will not review the report, except as given the opportunity to do so through the public review mechanism.

Key Informants must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the draft report are considered by the EPC in preparation of the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and may be published three months after the publication of the Evidence report.

Potential Reviewers must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest.Invited Peer Reviewers may not have any financial conflict of interest greater than $5,000.  Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.

This project was funded under Contract No. xxx-xxx from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The AHRQ Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

PUBMED

"Quality Indicators, Health Care"[Mesh] OR “Healthcare Quality Indicator” [Title/Abstract]  OR “Healthcare Quality Indicators” [Title/Abstract] OR “care indicator” [Title/Abstract] OR “care indicators” [Title/Abstract]  OR “care quality” [Title/Abstract] OR “Health metric” [Title/Abstract]  OR “Health metrics” [Title/Abstract]  OR “care metric” [Title/Abstract]  OR “care metrics” [Title/Abstract]  OR "Benchmarking"[Mesh] OR benchmarked[Title/Abstract] OR benchmarking[Title/Abstract] OR benchmark[Title/Abstract] OR benchmarks[Title/Abstract]  

AND framework[Title/Abstract] OR frameworks[Title/Abstract] OR conceptual model[Title/Abstract] OR logic model[Title/Abstract]