• We examined two Key Questions (KQs). KQ 1 explored the effect of healthcare algorithms on racial and ethnic disparities in access to care, quality of care, and health outcomes. KQ 2 identified strategies to mitigate racial and ethnic bias associated with algorithms.
• For KQ 1, we identified 17 studies examining the effect of 18 algorithms on racial and ethnic disparities in health and healthcare. Four of the 18 algorithms included race or ethnicity as an input variable. The most frequently examined algorithms are used (or, in a few instances, are suggested for use) to inform resource allocation in a crisis setting (e.g., crisis standards of care) (4 studies), guide emergency department (ED) care decisions (3 studies), determine eligibility for lung cancer screening (3 studies), and determine eligibility for prostate cancer screening (2 studies). None was a randomized controlled trial (RCT).
• KQ 1 studies found that algorithms may reduce racial and ethnic disparities (5 studies), perpetuate or exacerbate disparities (11 studies; 3 of 11 included an examination of methods to mitigate these disparities and thus addressed both KQ 1 and 2), or have no effect on disparities (1 study). Three of the four studies examining algorithms that included race and ethnicity as an input variable found that these algorithms actually or potentially reduced disparities. In some algorithms (e.g., revised Kidney Allocation System), race and ethnicity input variables were included specifically to address existing racial and ethnic disparities.
• Studies addressing KQ 2 examined strategies for mitigating racial and ethnic disparities associated with algorithms. We included 44 studies across a range of clinical applications, including measurement of kidney function and lung function; risk assessment for cardiovascular disease, stroke, lung cancer, opioid misuse, and postpartum depression; suitability for kidney and liver transplant; and anticoagulation titration.
• Six types of mitigation strategies were identified: removing a race or ethnicity input variable from the algorithm (24 studies); replacing race or another input variable with a different measure (5 studies); adding an input variable (9 studies); recalibrating the algorithm with a more representative patient population (4 studies); stratifying algorithms to assess Black and White patients separately (2 studies); and using different statistical techniques within algorithms (3 studies).
• Most studies that examined the impact of removing a race coefficient from a common kidney function measure (eGFR) found an increase in diagnoses of chronic and severe kidney disease in Black patients. This may lead to a decrease in disparities in both early nephrology referrals for chronic kidney disease and referrals for kidney transplant. Conversely, some studies demonstrated a potentially negative impact of removing the race coefficient on health and healthcare outcomes other than transplant eligibility among Black patients when removing the race coefficient (e.g., patients reclassified as having more severe kidney disease could be deemed ineligible, possibly inappropriately, for medication or enrollment in clinical trials).
• Algorithms are often developed by electronic health record vendors, payers, and health systems. Due to the propriety nature of these algorithms, little is known about the development approach and potential impact on racial and ethnic disparities.
• Awareness is low among patients, healthcare providers, payers, and policymakers of the potential for algorithms to affect racial and ethnic disparities.

Objectives. To examine the evidence on whether and how healthcare algorithms (including algorithm-informed decision tools) exacerbate, perpetuate, or reduce racial and ethnic disparities in access to healthcare, quality of care, and health outcomes, and examine strategies that mitigate racial and ethnic bias in the development and use of algorithms.

Data sources. We searched published and grey literature for relevant studies published between January 2011 and February 2023. Based on expert guidance, we determined that earlier articles are unlikely to reflect current algorithms. We also hand-searched reference lists of relevant studies and reviewed suggestions from experts and stakeholders.

Review methods. Searches identified 11,500 unique records. Using predefined criteria and dual review, we screened and selected studies to assess one or both Key Questions (KQs): (1) the effect of algorithms on racial and ethnic disparities in health and healthcare outcomes and (2) the effect of strategies or approaches to mitigate racial and ethnic bias in the development, validation, dissemination, and implementation of algorithms. Outcomes of interest included access to healthcare, quality of care, and health outcomes. We assessed studies’ methodologic risk of bias (ROB) using the ROBINS-I tool and piloted an appraisal supplement to assess racial and ethnic equity-related ROB. We completed a narrative synthesis and cataloged study characteristics and outcome data. We also examined four Contextual Questions (CQs) designed to explore the context and capture insights on practical aspects of potential algorithmic bias. CQ 1 examines the problem’s scope within healthcare. CQ 2 describes recently emerging standards and guidance on how racial and ethnic bias can be prevented or mitigated during algorithm development and deployment. CQ 3 explores stakeholder awareness and perspectives about the interaction of algorithms and racial and ethnic disparities in health and healthcare. We addressed these CQs through supplemental literature reviews and conversations with experts and key stakeholders. For CQ 4, we conducted an in-depth analysis of a sample of six algorithms that have not been widely evaluated before in the published literature to better understand how their design and implementation might contribute to disparities.

Results. Fifty-eight studies met inclusion criteria, of which three were included for both KQs. One study was a randomized controlled trial, and all others used cohort, pre-post, or modeling approaches. The studies included numerous types of clinical assessments: need for intensive care or high-risk care management; measurement of kidney or lung function; suitability for kidney or lung transplant; risk of cardiovascular disease, stroke, lung cancer, prostate cancer, postpartum depression, or opioid misuse; and warfarin dosing. We found evidence suggesting that algorithms may: (a) reduce disparities (i.e., revised Kidney Allocation System, prostate cancer screening tools); (b) perpetuate or exacerbate disparities (e.g., estimated glomerular filtration rate [eGFR] for kidney function measurement, cardiovascular disease risk assessments); and/or (c) have no effect on racial or ethnic disparities. Algorithms for which mitigation strategies were identified are included in KQ 2. We identified six types of strategies often used to mitigate the potential of algorithms to contribute to disparities: removing an input variable; replacing a variable; adding one or more variables; changing or diversifying the racial and ethnic composition of the patient population used to train or validate a model; creating separate algorithms or thresholds for different populations; and modifying the statistical or analytic techniques used by an algorithm. Most mitigation efforts improved proximal outcomes (e.g., algorithmic calibration) for targeted populations, but it is more challenging to infer or extrapolate effects on longer term outcomes, such as racial and ethnic disparities. The scope of racial and ethnic bias related to algorithms and their application is difficult to quantify, but it clearly extends across the spectrum of medicine. Regulatory, professional, and corporate stakeholders are undertaking numerous efforts to develop standards for algorithms, often emphasizing the need for transparency, accountability, and representativeness.

Conclusions. Algorithms have been shown to potentially perpetuate, exacerbate, and sometimes reduce racial and ethnic disparities. Disparities were reduced when race and ethnicity were incorporated into an algorithm to intentionally tackle known racial and ethnic disparities in resource allocation (e.g., kidney transplant allocation) or disparities in care (e.g., prostate cancer screening that historically led to Black men receiving more low-yield biopsies). It is important to note that in such cases the rationale for using race and ethnicity was clearly delineated and did not conflate race and ethnicity with ancestry and/or genetic predisposition. However, when algorithms include race and ethnicity without clear rationale, they may perpetuate the incorrect notion that race is a biologic construct and contribute to disparities. Finally, some algorithms may reduce or perpetuate disparities without containing race and ethnicity as an input. Several modeling studies showed that applying algorithms out of context of original development (e.g., illness severity scores used for crisis standards of care) could perpetuate or exacerbate disparities. On the other hand, algorithms may also reduce disparities by standardizing care and reducing opportunities for implicit bias (e.g., Lung Allocation Score for lung transplantation). Several mitigation strategies have been shown to potentially reduce the contribution of algorithms to racial and ethnic disparities. Results of mitigation efforts are highly context specific, relating to unique combinations of algorithm, clinical condition, population, setting, and outcomes. Important future steps include increasing transparency in algorithm development and implementation, increasing diversity of research and leadership teams, engaging diverse patient and community groups in the development to implementation lifecycle, promoting stakeholder awareness (including patients) of potential algorithmic risk, and investing in further research to assess the real-world effect of algorithms on racial and ethnic disparities before widespread implementation.

Tipton K, Leas BF, Flores E, Jepson C, Aysola J, Cohen J, Harhay M, Schmidt H, Weissman G, Treadwell J, Mull NK, Siddique SM. Impact of Healthcare Algorithms on Racial and Ethnic Disparities in Health and Healthcare. Comparative Effectiveness Review No. 268. (Prepared by the ECRI-Penn Medicine Evidence-based Practice Center under Contract No. 75Q80120D00002.) AHRQ Publication No. 24-EHC004. Rockville, MD: Agency for Healthcare Research and Quality; December 2023. Addendum April 2024. DOI: https://doi.org/10.23970/AHRQEPCCER268. Posted final reports are located on the Effective Health Care Program search page.