Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care
  • Home
  • Search for Research Summaries, Reviews, and Reports

EHC Component

  • EPC Project

Topic Title

  • Methods for Benefit and Harm Assessment in Systematic Reviews

Research Report - Final – Feb. 13, 2014 (Update)

Evaluation of the Benefits and Harms of Aspirin for Primary Prevention of Cardiovascular Events: A Comparison of Quantitative Approaches


People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us.

Structured Abstract


Prior work has described various quantitative approaches to the assessment of benefits and harms of medical interventions. Researchers rarely use these approaches in the context of a systematic review.

Our objectives were to illustrate two quantitative approaches to assessing benefits and harms in the context of a systematic review, and to determine the methodological challenges of applying these approaches in a systematic review.


We compared the number-needed-to-treat (NNT) and number-needed-to-harm (NNH) approach and the Gail/National Cancer Institute (NCI) approach for assessing the benefits (prevention of myocardial infarction [MI] and ischemic stroke) and harms (excess of hemorrhagic stroke and major gastrointestinal [GI] bleeds) of aspirin for primary prevention of cardiovascular events. We based our main analyses for these two approaches on the treatment effects from a meta-analysis of large primary prevention trials, and the incidence rates from observational studies. We focused on observational studies that were most applicable to our target population--aged 50 to 84 years, living in the United States without evidence of cardiovascular disease or stroke. We obtained relative weights denoting the relative importance of different outcomes (required by the Gail/NCI approach) from literature sources. These sources weighted major stroke nearly twice as much as MI and nearly eight times as much as major GI bleeds.


The NNT and NNH for aspirin declined with increasing age because of the increase in baseline incidence rates for all outcomes across age categories as obtained from observational studies. For example, in men aged 45–54, the NNT was 1,786 person-years of treatment to prevent one MI, and the NNH was 1,344 person-years of treatment to induce one major GI bleed (which corresponds to 5.6 MI prevented and 57.4 GI bleeds induced if 1,000 people are treated with aspirin for 10 years, compared with no aspirin use). For men aged 75–84, the NNT was 511 to prevent one MI and the NNH was 202 to induce one major GI bleed. A sensitivity analysis that considered different baseline incidence rates from randomized trials showed a much higher NNH for GI bleeds because the baseline incidence rate of that outcome was 2–3 times lower than in observational studies.

When we used relative weights, the Gail/NCI approach showed that aspirin caused more benefit than harm in all age categories of men and women. When we weighted outcomes equally in a sensitivity analysis, the harm from aspirin was greater compared with the main analysis because of greater relative weight for GI bleeds. When we weighted stroke as a very important outcome (weight of 1), MI as an important outcome (weight of 0.5), and GI bleed as an unimportant outcome (weight of 0), aspirin was associated with net benefit for all sex and age categories.

When comparing the two approaches in terms of estimates for a single outcome, we found comparable results for the number of people who would have a benefit or harm from treatment as long as the baseline incidence rates and the competing risk (all-cause mortality) were small. When the impact of the competing risk was larger, we found substantial differences between the NNT and NNH and Gail/NCI approaches, even though the baseline incidence rates and treatment effects used were identical.


The assessment of benefits and harms requires careful selection and integration of data from disparate sources, including baseline risks of events without treatment, the effects of treatments on various outcomes, and relative weights of these outcomes. We have illustrated that quantitative approaches are feasible in a specific decisionmaking context--using data from a systematic review of aspirin for primary prevention. Quantitative approaches can yield different results even if input data for baseline risks and treatment effects are identical. Quantitative approaches can be particularly valuable in demonstrating how the expected balance of benefits and harms depends on assumptions about the relative weights of different outcomes.

Journal Publications

Puhan MA, Singh S, Weiss CO, et al. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol 2012 Nov 19;12:173.