Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care

Back to "About This Webinar"

Using Deliberative Methods to Engage Patients, Consumers, and the Public

Webinar Transcript, December 6, 2011

Slide 1

Using Deliberative Methods to Engage Patients, Consumers, and the Public
Agency for Healthcare Research and Quality
Community Forum
December 6, 2011

Speaker notes: Welcome, everyone, to this webinar about using deliberative methods to engage patients, consumers, and the public

Slide 2

Purpose and objectives

  • Purpose
    • Describe deliberative methods
    • Explore how deliberative methods can be used in your work
  • At the end of this, you should be able to:
    • Identify situations appropriate for deliberative methods
    • Develop the questions that can be addressed through deliberative methods
    • Understand how deliberative methods can be used

Speaker Notes: First, we want to go over the purpose of today’s session.  We will describe deliberative methods – providing key characteristics of deliberative methods and helping you to recognize when deliberative methods are appropriate to use.  We will also explore how deliberative methods can be used in the work that you or your organization does to reach patients, consumers, caregivers, and lay members of the public.

At the end of this session, you should be able to:

  • Identify situations that warrant deliberative methods
  • Develop questions that can be addressed through deliberative methods
  • Understand how deliberative methods can be used

Slide 3

Presenter introductions

  • Kristin L. Carman, PhD, Managing Director, American Institutes for Research (AIR)
  • Ela Pathak-Sen, Director, Commotion
  • Jessica Waddell, MPH, Researcher, AIR
  • Marge Ginsburg, MPH, Director, Center for Healthcare Decisions

Images: Each presenter has a corresponding photograph 
Speaker notes: Now, I want to introduce myself and my colleagues who will talk with you today.

I am Kristin Carman, Managing Director at the American Institutes for Research.  Thank you all for joining.  We’re really excited that you are participating.  We have 253 people on the line, coming from the Food and Drug Administration, Fraser Health, Institute of Medicine, National Health Council, National Partnership for Women and Families, Ovarian Cancer Advocacy Alliance, Pacific Business Group on Health, and the University of North Carolina, among many others.

I want to introduce my co-presenters: Ela Pathak-Sen, Jessica Waddell, and Marge Ginsburg.  I also want to recognize Jess Fernandez, our research assistant, who has helped us tremendously in our preparation.

Also, Joanna Siegel, our Project Office at AHRQ, is on the line and has contributed a great deal to this presentation.

Slide 4

Reason for this work

  • Agency for Healthcare Research and Quality’s 3-year initiative called Community Forum
  • Led by the American Institutes for Research (AIR) with key partners
  • Major part of this project is to:

Advance the use of deliberative methods for obtaining input from members of the public on a health research topic
Speaker notes: This webinar is part of a large, three-year project, funded by the Agency for Healthcare Research and Quality (AHRQ) and led by the American Institutes for Research (AIR).

Initiated in August 2010, this project entails two key parts:

    • To expand stakeholder engagement in AHRQ’s Effective Health Care Program,
    • To advance the use of public deliberation or deliberative methods for obtaining lay public input a health research topic.

This second part of the project seeks to expand the evidence base about public deliberation and garner public input on a health topic related to comparative effectiveness research to inform AHRQ’s EHC Program.  To do so, we are conducting a randomized controlled experiment comparing five deliberative methods to each other and to a control group.  This webinar focuses on the second part – regarding what we have learned about deliberative methods to date.

The basis for this presentation is an extensive literature review of public deliberation, including international applications and applications in fields such as health, bioethics, education, politics, and environmental policy, conducted in fall 2010 – early 2011.   The literature review included peer-reviewed and grey literature, and key informant interviews.

Further, as part of the randomized control trial (RCT) we will be conducting, we have undergone an extensive planning process over the past year, informed by the literature review, input from our Technical Expert Panel, and AHRQ, and gleaned lessons that are immediately applicable to others’ work in deliberative methods, which we share today. 

Slide 5


  • Overview of deliberative methods
  • Application to health
  • Examples from the literature
  • IOM committee report on essential benefits
  • Interactive exercise
  • Discussion and Q & A

Speaker notes: As a reminder, we will discuss, first, an overview of deliberative methods – covering defining features of deliberation, the issues that are appropriate for deliberative methods, and the best practices in designing deliberative methods.

Next, we will highlight the application of deliberative methods to health through some key examples. 

Then, we will present a few real-world applications of deliberative methods in health.

Finally, we will move into an interactive exercise around starting a deliberative process.  Using some of the issues your organizations shared with us prior to the webinar, we will discuss how deliberative methods can be used to address these issues.

Slide 6

Overview of Deliberation
Speaker notes: First, we will start with an overview of deliberative methods.

Slide 7

What is deliberation

  • Convening of “mini publics” (Fishkin 2009) …
  • …“to weigh carefully the consequences of various options for action and the views of others” (Matthews 1994)
  • Provides opportunity to weigh the “principles and values involved as well as the circumstances and consequences” of the topic of debate (Gracia 2003)

Speaker notes: So, what is deliberation?  Public deliberation is founded on deliberative democratic theory.  It entails, first, bringing lay members of the public together – in “mini publics” (small groups of people who can represent the broader public). Deliberative methods – which are distinct applications of public deliberation – can include those individuals who are disproportionately or highly affected by an issue such as patients.  If you include individuals who are particularly affected by an issue, you need to ask them to consider their views in relation to everyone else – to encourage a societal view.

Together, they learn about and discuss an issue – this has been referred to as the careful weighing of the sides of an issue, considering the various principles and values involved along with the consequences of any action to society overall.

Deliberative democratic theory specifies that such interaction results in informed public opinion, eliciting perspectives of individuals once they have had the opportunity to carefully think about an issue.  The key is that such a process elicits how people actually think and feel about an issue once they learn more about it and hear from others’ perspectives and that this resulting opinion is more stable and representative of how the public feels than opinions that would be garnered through more traditional forms of consultation such as opinion polling.

The goal of deliberation is fundamentally to encourage people to learn and think openly about an issue, hearing others’ perspectives, so that in the end, conveners or sponsors of deliberation learn from the participants.

Slide 8

Goals of deliberative methods

  •  To provide decision makers with understanding of public values relevant to complex issues
  • To influence change at policy or program level
  • To expand participants’ knowledge and insight on an issue
  • To increase participants’ civic engagement and willingness to participate

Speaker notes: Next, what are the goals for deliberation?  Why would you do this?  Deliberative methods can provide decision-makers with an understanding of public values relevant to complex issues.  One way it does this is by including diverse perspectives, including those who may be disproportionately affected by an issue or who are traditionally excluded from such processes.

It can influence change at a policy or program level.  Having the public’s input can be powerful vis-à-vis working with other stakeholders, which may motivate change.   There is some – albeit limited – evidence that public deliberation affects policy or program level changes.  The key here is the early engagement of and delivery of the participants’ input to decision-makers.

Further, deliberation can expand participants’ knowledge and insight on an issue.  Through the educational process, deliberative processes can reveal information gaps, misinformation, and misunderstandings—which are addressed through the deliberative process. Based on evaluations of public deliberation found in the literature review, deliberation was found to impact participants’ knowledge and attitudes on an issue.  For example, participants may demonstrate that they gained additional knowledge about the topic and their attitudes may change.

The literature review also revealed that previous evaluations have found that deliberation increases participants’ civic engagement and willingness to participate in civic activities in the future.

Slide 9

Where deliberative methods fit

  • Intensity of involvement
  • Image: Arrow pointing to the right with three boxes on top of it. Each box from left to right has words in it: “Opinion polling,” “Focus groups,” “Deliberative methods,” respectively.
  • Informed input

Speaker notes: We wanted to give you an idea of where deliberation fits in with other public consultation methods—how does it compare in terms of intensity of involvement and the resulting input from the participants. 

While there are many other activities along this spectrum and arguably more intensive activities that follow deliberative methods, this diagram is intended to give you a high-level understanding of where deliberation fits in. 

Generally speaking, deliberative methods are approaches to public consultation or political participation that are more intensive than more traditional approaches such as opinion polling or focus groups.

Opinion polling—on the far left—is a common way in which the public gives feedback or provides input.  However, opinion polling elicits top-of-mind responses – deliberation, alternatively, elicits more stable, considered input because the process iteratively involves education and conversation with diverse individuals, enabling individuals to reevaluate information and their positions.

Moving to the right, deliberative methods—just like focus groups—involve the convening of participants in group discussions.  However, deliberative methods are more intense than traditional focus groups, as we will highlight in the next slide.

Slide 10

Components of deliberative methods

  • Image: Arrow pointing to the right with three boxes. Each box from left to right has words in it: “Opinion polling,” “Focus groups,” “Deliberative methods,” respectively.
  • Convening of groups
    • Discussion
      • Education
      • Reason-giving and debate
      • Societal perspective

Speaker notes: Both focus groups and deliberative methods involve the convening of groups and discussion.  However, deliberation is more intensive because: (1) it consists of an educational component to ensure that all participants are starting from a shared level of knowledge and understanding, (2) emphasizes a reason-based dialogue and debate, and (3) aims to elicit a community-based perspective and thus orient the conversation to societal-level thinking. 

Further, education is a defining feature of deliberation, as I mentioned previously. 

Additionally, because a key goal of deliberation is to encourage participants to hear from others on the issue, deliberation involves active debate. The idea is that each participant’s points of view will be expanded through this process.  It is also in the debate that the various values and ethics involved will emerge from the participants’ perspectives. 

Another element of deliberation is reason-giving – which is the act of providing the rationale for your statements.  This may be a new exercise for many participants, but it is essential to deliberative democratic theory.  Requiring that participants share their reasoning not only enriches their input but also enables other participants to more fully respond to their comments.  It also instills accountability – people are asked to explain their reasoning, not just make statements. 

Finally, deliberative methods involve asking participants to assume a societal perspective (sometimes, as societal decision-makers).  The idea here is that deliberation is intended to provide insight into how people feel about the issue when they consider everyone and the possible consequences that will affect society.

Slide 11

Characteristics of deliberative topics

  • Values-based or ethical dilemmas
  • Social/affecting common good
  • Cannot be resolved through technical or scientific information alone
  • Controversial but opportunity for common ground
  • Timely and relevant

Speaker notes: Here, we highlight the criteria for selecting topics for deliberation.

Topics appropriate for deliberative methods consist of values-based or ethical dilemmas—often where values are potentially in conflict or various ethical/moral issues need to be balanced and reconciled.  An example issue may be the decision between incenting public transportation vs. deterring personal transportation; this issue involves concerns for environment, sustainability of public transportation system, equity considerations (e.g., do all communities have access to public transportation?), traffic considerations, toll on infrastructure (i.e., more cars=more toll), personal convenience, and the impact on the auto industry, among other considerations.

Topics appropriate for deliberation are also social issues that affect the common good or the community – there needs to be a stake for a member of the public.  For example, building on the same issue – there is a stake for everyone in environmental wellbeing and the availability of public transportation, so this issue is appropriate for deliberation.

Further, appropriate topics or issues cannot be resolved through technical or scientific information alone.  The key is that members of the public, patients, or consumers are engaged for their “life expertise” – their perspectives and experiences are important for resolving or shedding light on the issue.  Extending the same example further – this issue requires consideration of multiple stakeholders' perspectives, most importantly the public's.  An issue such as inspecting buses for safety is not a deliberative topic because there is a technical answer.  The key is whether other perspectives and multiple pieces of information – such as values – are relevant.

Topics should also be controversial but provide opportunity for common ground – for example, retirement benefits.  People may have deeply held, emotionally charged views on this issue about responsibility, entitlement, method of financing, etc. of retirement benefits; however, despite these strong feelings, there is an opportunity to come together because society has a stake in how well people are prepared for retirement.  Issues such as abortion in the U.S. may not be best addressed through deliberation because the feelings are so deeply entrenched that movement toward a shared goal may not be feasible.

Finally, issues should be timely and relevant to current public policy or decisions so that participant input will be used to affect policy, programs, or other decision-making.  For example, deliberating on the War in Iraq would not be timely now – but discussing troop withdrawal, Afghanistan, Pakistan, and/or Iran is relevant to current policy and decision making.  You could still deliberate on an issue about which policymakers or other decision-makers have made a decision but you think a public voice could counter the decision or be used in future decisions if the issue is a persistent societal one (e.g., how we finance public education, how we subsidize agriculture).

Slide 12

How deliberative methods vary

  • Length, duration
  • Group size, participant sample
  • Recruitment method
  • Structure (e.g., breakout groups, interrupted)
  • Education, use of experts
  • Mode (online, in-person)
  • Facilitation
  • Consensus as goal

Speaker notes: There are many distinct deliberative methods with defining features in the field, including citizens’ juries, consensus conferences, and Deliberative Polling, among others, and many examples of hybrid methods – which borrow key features from the distinct methods.  Because a discussion of the methods can be laborious, we wanted to, instead, give you a sense of how deliberative methods vary.

They can vary in length and duration – with ranges from 2 hours to a standing commitment of 3 years (with intermittent meetings during this period).

They can also vary by recruitment method – some methods use convenience sampling; some use random sampling.  This depends on whether a representative sample of the lay public is a goal.

Methods also vary by group size – ranging from groups of about 12 to upwards of 500. 

You would select among these features based on resources, goals, desire for representativeness, and nature of topic (for instance, highly emotionally or politically charged issues may be better for an interrupted structure – in which participants could have a break, whereas a complex, technical issue may warrant the use of multiple experts). 

Now, I will transition to Ela, who will tell you more about how to best design and implement these methods.

Slide 13

Steps to designing deliberative methods

  • Identify and define issue needing input
  • Engage the decision makers for whom results are intended
  • Set goals for deliberation
  • Determine participant population (e.g., lay public, patient group)
  • Select deliberative process or create hybrid
  • Determine evaluation approaches

Speaker notes: There are a few key steps to designing deliberative methods including:

  • First, identifying and defining the issue needing input – here is where you think about the issues that you want to bring to your constituent groups or community members. 
  • Second, you should consider involving policy and decision makers in the process.  You could engage them early in the process – during the formative stages – or consider presenting/sharing the findings from the deliberation with them to see how they would use the input.  Early engagement is ideal.
  • Third, you should set goals for the deliberation.  This often entails clarifying how you think your target participant group could make a difference, what information you would like to obtain, and what intended outcomes for the engagement you would like to see.
  • Next, you want to consider your target participant population – who do you want to engage. 
  • Then, you should select the deliberative method or develop a method.  Key features of methods include the length, mode (in person or online), group size, etc.  Key features of methods include the length, mode (in person or online), group size, as I just mentioned.  When you select a method, you are also making decisions about the nature of the educational component, style of facilitation (e.g., active/passive), and whether you will use experts.
  • Finally, you need to establish any evaluation approaches, such as pre/post surveys.  These may include assessing the fidelity of the implementation to the method.

Slide 14

Best practices

  • Early involvement of audience for participant input
  • Diversity
  • Balance
  • Transparency
  • Clarity
  • Adapt to context

Speaker notes: Based on the literature review, best practices in designing deliberative methods include, first the early involvement of the audience for the public’s input, as we have discussed.  This is key for both assuring participants that their input will be used and valued and also for facilitating the uptake and use of the participants’ input.  This is especially important when deliberation may be a new method to policy or decision makers.

Further, diversity is key.  It is important to consider diversity in all aspects of the design – including the participant recruitment, group composition, educational materials and modes (e.g., written, audio/video), languages used and available, experts’ identities and perspectives, and facilitators, among other considerations.  Since diversity is essential for promoting one of the goals of deliberation, i.e., rich debate, designing deliberative methods so that they include diversity of perspectives is crucial for promoting good outcomes.

Balance in the perspectives presented is also important.  This means that multiple sides of the issue are presented and given their due time.  For instance, regarding an issue such as mandatory vaccination, you would want to include such experts as public health practitioners, pediatricians, environmental health experts, ethicists, and proponents of individual liberties.  The key is to make sure that the educational process, which may include experts, is fair to each side of the issue.  This component not only bolsters the richness of the debate but also promotes the transparency and legitimacy of the process.

A fourth best practice is transparency of process.  This entails sharing the process with participants, policy and decision makers, and other external observers.  You will want to make sure that participants understand why they are being involved, what their role is and what is being asked of them, how their input will be used (if known), and what follow up with them will be like, if there will be any at all.  In addition, you will want to make sure that your audience for the public’s input understands the educational process and how input was elicited.

Clarity is also important.  As highlighted when discussing transparency, it is important to be clear with participants regarding their role and what they will be asked to do.

Finally, it is important to adapt your methods, materials, and overall approach to the current political, social, and cultural contexts.  Essentially, keeping an eye on the environmental factors (e.g., media coverage, an election) that may influence how participants feel about an issue is important for both designing and implementing your method as well as any evaluation of it.

Slide 15

Applications of deliberative methods to health care

Speaker notes: Now, we will consider how deliberative methods are relevant and applicable to healthcare.

Slide 16

How deliberative methods can help inform health care and health research

Example issues How deliberation can help

Priority setting comparative effectiveness research topics

Balancing of values, prioritization of topics

Organ shortages for transplantation

Guidance on societal views of ethics involved

Racial/ethnic disparities in maternal  and infant mortality

Propose solutions and establish goals for improvement

Quarantine during public health emergencies

Balancing of societal wellbeing vs. individual rights

Speaker notes: Here, we have listed a few example issues in public health or healthcare that are applicable to public deliberation on the left.  And on the right hand side, we have listed how deliberation could help. 

For example, priority-setting for comparative effectiveness research.  The American Recovery and Reinvestment Act allocated substantial funding to the conduct of CER and development of research infrastructure to support this important research in the future.  There are many conditions, treatments, and subpopulations that warrant CER studies; however, with limited research resources and time, using a deliberative process can reveal the public’s values in terms of research and which conditions, treatments, or populations they feel should be prioritized – and why.  Ultimately, deliberation on this topic could result in a prioritization of topics.

Another example is the ethical dilemma posed by organ shortages for transplantation. In the literature, we found several examples of public deliberation being used for bioethical topics such as this one.  Deliberation could provide guidance on societal views on the ethics involved and how they should be balanced.

Deliberation could also be used to address health disparities issues such as racial/ethnic disparities in maternal and infant mortality.  The literature included some examples of deliberation in which participants were asked to consider a health problem and develop ideas for solutions or goals for addressing it.  Deliberation on such topics raises awareness and can help generate ideas that can inform public health policy or programs.

Public health inherently involves dilemmas such as the fundamental tension between societal wellbeing and individual rights.  Because of this, deliberation can be used in emergency preparedness.  Several examples of deliberation in the literature involved public health issues.  Input on these issues can inform public health planning.

Slide 17

Real-world applications of deliberative methods

  • Two examples from the literature
    • Surrogate consent for research participation
    • Priority-setting social and health interventions
  • IOM’s Committee on Determination of Essential Health Benefits, Oct. 2011

Speaker notes: To make all of this explication more concrete, we are going to highlight several key examples of deliberation – two of which come from the literature.  The first addresses surrogate consent for research participation. 
The second study asked participants to prioritize social and health interventions based on the interventions they believed would have the biggest impact on health.
Finally, Marge Ginsburg will share some of her experience and insights from serving on the Institute of Medicine’s Committee on the Determination of Essential Health Benefits, who recommended that public deliberation be used in determining the benefits.

Slide 18

Examples from the literature

Speaker notes: Now, we will cover some examples from the literature review.

Slide 19

Deliberative method for assessing surrogate consent

  • Issue: Can family members provide surrogate consent for research participation for individuals with dementia?
  • Researchers: led by University of Michigan with funding from National Institute on Aging
  • Goal of deliberation:
    • Assess how participants view surrogate consent for research participation for individuals with dementia
    • Evaluation:  Determine whether deliberation affected caregivers’ views of surrogate consent

Speaker notes : The first study we will highlight entailed a deliberative method for assessing surrogate consent policies.  This study sought to address the issue of surrogate consent for research participation and determine the effect of deliberation on people’s views of surrogate consent. As Kristin described, it is key that issues for deliberation are complex and cannot be resolved through technical information alone.  In the U.S., regulations allow research with incapacitated adults based on consent by their legally authorized representatives. However, states have tended not to define what a legally authorized representative is.  Further, there is a lack of consensus regarding the special protections that are needed for subjects enrolled in surrogate-based research.  Since this issue involves scientific, regulatory, and ethical complexities, it is well-suited for deliberation.

This study was led by Scott Kim and other researchers at the University of Michigan with a team from Columbia University, New York Psychiatric Institute, Ann Arbor Veterans Administration Medical Center, and Queensland University of Technology and conducted with funding from the National Institute on Aging within the NIH.

The participant sample was selected through community-based recruitment and consisted of caregivers or primary decision-makers for persons with dementia.

The deliberative method consisted of one-time, in-person sessions with breakout groups.  In the educational component, participants learned about Alzheimer’s disease, the clinical research for AD, and the ethics of surrogate-based research.

As Ela mentioned, you’ll want to determine whether you will evaluate your approach and how you will do so.  The researchers here employed a randomized controlled evaluation to determine whether deliberation affected the caregivers’ views of surrogate consent and administered a pre/post survey at three points in time.  The survey assessed attitudes toward surrogate-based research.

The evaluation found that deliberation participants increased their endorsement of a societal policy for surrogate-based research, and this increase was sustained one month after deliberation.  A key question in the literature is how long the effects of deliberation last – so demonstrating a sustained opinion change is important. 

Slide 20

Deliberative method for prioritization of interventions

  • Issue: How would you decide which social or health services to provide to improve health?
  • Researchers: National Institutes of Health, Howard University, and D.C. Department of Health
  • Goals:
    • Learn how participants prioritize social or health services to improve health and understand their reasoning
  • Evaluation: assess whether deliberation affected participants’ knowledge on the determinants of health

Speaker notes: The second study I will highlight was conducted by Julianna Pesce and others at the NIH, Howard University, and the District of Columbia Department of Health to learn how participants would prioritize social or health services for health improvement through a deliberative process.  Researchers wanted to understand how participants would prioritize services and programs to improve health – and their underlying reasoning.

Participants received educational materials including descriptions of 25 interventions.  Participants learned about social determinants of health – or the environmental and contextual factors, such as where we live, that affect health outcomes.  Researchers were interested in learning how this information would influence how participants prioritized interventions.  Participants were given a budget and could see the costs of each intervention.

An evaluation was conducted using pre/post surveys and a qualitative analysis of the deliberations. 

Slide 21

Deliberative method for prioritization of interventions cont.

  • Participants: Spanish or English speakers with income 200% of FPL
  • Method: REACH, priority setting game
  • 4 rounds of priority setting
    • Self, family
    • Neighborhood
    • Entire city
    • Self, family again

Speaker notes: Participants comprised community members in D.C., with incomes under 200% of Federal Poverty Line (FPL) who were Spanish or English speaking.  In total, 431 people were recruited through newspaper advertisements and flyers.  Groups were conducted in Spanish or English. 

The deliberative method utilized an approach called “REACH” which stands for Reaching Economic Alternatives that Contribute to Health.  This model is based on Choosing Healthplans All Together (CHAT), which is a game that serves as an educational and deliberation tool. CHAT was designed to elicit public priorities regarding health insurance benefits; it has been used for employer-sponsored insurance and public programs.  Participants were asked to design a benefit package of social or health interventions to improve health. 

Interestingly, there were four rounds of priority-setting. First, participants were asked to think about themselves and their families when ranking the interventions.  Second, they were asked to think about the neighborhood (while working in small groups), and then they were asked to deliberate in a larger group and think about the entire city.  In fourth round, participants were again asked to think about themselves and their families to see if their personal priorities changed. As we’ve discussed, deliberation is intended to move participants to a societal perspective.

Participants were asked to assess their experience of participating, and most strongly agreed that the exercise was informative.  Participants also demonstrated increased knowledge about social determinants of health.  Participants believed that health was related to quality of insurance, lifestyle, income, and neighborhood factors more strongly after the deliberation.  Participants prioritized health insurance, housing, dental care, and job training above other health/social interventions.

This study demonstrated that public deliberation could be used to educate participants about social determinants of health and to engage participants in a priority-setting exercise to learn more about which social or health programs and services are the most valued.

Now, I will transition to Marge, who will talk about her experiences on the IOM committee on the Determination of Essential Health Benefits.

Slide 22

IOM Committee
Report on Essential Benefits

Identifying principles and criteria
Speaker notes: Many of you may be aware that, when the Patient Protection and Affordable Care Act (ACA) was passed in 2010, the central part of this legislation was to bring health insurance coverage to millions of Americans. One of the big questions was what this new coverage should include.  Since all public and private health insurance goes through some process of determining what would be a covered benefit and what would not, the ACA assigned this task to the Department of HHS. 

Last year, Secretary Sebelius turned to the Institute of Medicine, asking that a committee be established to help define the principles and criteria that can provide direction on what the essential benefits package (EHB) should include. The reference to principles and criteria is a strong clue that the concept of ‘essential’ does not mean a simple sum of ‘all medical problems’ plus ‘everything that medicine has to offer.’  Principles and criteria suggest judgment, standards and values: attributes that are not solely the purview of providers or policymakers. 

The IOM report was released in early October. Included in the committee’s report are very specific recommendations about public deliberation as a component of the process of defining the ACA’s essential health benefits. 

I was on the IOM committee and will take a few minutes to note how the committee envisioned using public deliberation and why the committee saw this as an important aspect for determining the health benefits that will impact so many Americans. 

However, before answering the question, "How can the public voice help inform this task?” the first step is to examine the task itself: what are the ways that society can establish responsible boundaries of coverage?  Once those are identified, then one can ask: is there something we can learn from the public about those variables that might inform the decisions that HHS must make? 

Slide 23

Coverage elements to consider in determining ‘essential benefits’

  • Extent of medical conditions and treatments
  • Types and circumstances of patient cost-sharing
  • Pre-approvals and other clinical oversight
  • Limits on quantity of services
  • Standards of clinical effectiveness

IOM committee: the role of societal values
Speaker notes: There are several ways to define the boundaries of healthcare coverage in designing a benefits package.

The IOM committee did not have the authority to go beyond that task – so we couldn’t simply declare that all pharmaceutical companies must lower their drug prices by 50% or that all plan members must enroll in accountable care organizations.

Rather, the world of benefits design required consideration of the elements of coverage like the ones on this list. The committee recognized that, like all public and private sector health plans, there are a variety of ways to help keep coverage responsible and affordable. The most common ones are specified here: 1) extent of medical conditions and treatments, 2) types and circumstances of patient cost-sharing, 3) pre-approvals and other clinical oversight, 4) limits on quantity of services, and 5) standards of clinical effectiveness.

While the ACA specified that this EHB be based on a ‘typical health insurance coverage,’ how these elements are incorporated in a typical plan vary greatly from health plan to health plan. 

The IOM committee recognized that seeking the best balance among these elements could be informed by societal values that are identified through a deliberative process.

Slide 24

Examples of deliberative questions

  •  What are the characteristics of a medical problem that make it especially critical for coverage?
  • Are boundaries like ‘pre-approvals’ easier to accept than others?  Why?
  • When treatment effectiveness is minimal, what is a fair approach to coverage?

Speaker notes: So in a deliberative session, what questions might be central to the discussion?

    • What are the characteristics of the medical problem that make it especially critical for coverage?
    • Are boundaries like pre-approvals easier to accept than others? Why?
    • When treatment effectiveness is minimal, what is a fair approach to coverage?

Rather than asking these questions directly, a deliberative process allows people to think through and react to healthcare situations that they are familiar with – but may not have thought too deeply about their importance or relevance to coverage. The answers to these questions are the end-product of such a discussion.

Most important, through deliberation, people are able to respond to situations as ‘social decision-makers’ – if we, as a group of people with a stake in this issue are providing guidance, what is the fairest approach that takes into account my needs and the needs of others?

Slide 25

IOM: using public deliberation to inform coverage decisions

  • When public deliberation is recommended
    • Initial coverage details
    • State waivers
    • Updating the benefits package
  • The rationale for these inclusions
    • No easy answers
    • Social decisions

Speaker notes: The IOM Committee report is structured around five recommendations, and public deliberation plays a meaningful role in three of them:

  • Initial coverage details (prior to the initial determination)
  • State waivers (when states request a waiver to establish their own design)
  • Updating the benefits package (when HHS revises the benefits based on changes in medical science and research).

Why so much attention to this?  The committee recognized that virtually everything in healthcare has value to some people.  Even medical problems that most may dismiss as inconsequential or medical treatments that could be regarded as marginally helpful – those too represent ‘value’ to some patients and physicians. Yet health insurance does not pay for everything and the public and policymakers know this. 

What public deliberation will do is help answer the question: what is it that society owes its members and what should members be responsible for themselves? The IOM report recommends that these deliberative sessions be comprised of community members who will be recipients of the EHB – not to answer the question in terms of ‘what I want for myself’ but rather in terms of ‘what do all of us regard as a fair, responsible and affordable benefits package when trade-offs are necessary.’

I think the issue most problematic for policymakers is why a deliberative process is needed when many assume, understandably, that it is elected representatives, consumer advocates and the general public itself that should be able to speak on behalf of the best interests of the population at large.  What does this get us that is different and that is worthwhile?

The simplest answer lies in the process of deliberation itself: all deliberative endeavors involve situations where there are competing priorities.  Where answers are obvious, there is no need for deliberation.  It’s when there is more than one way to approach a problem or resolve a conflict that lends itself to deliberation. The other is the concept of a societal decision.  Insurance by its nature is dependent on its users accepting the boundaries of coverage. That also suggests that its users be the ones to influence what those boundaries are.

Our common vehicles for public participation – town hall meetings, etc. – usually ask the question ‘what do you want?’ not ‘what are your priorities?’ Implied in priority-setting is an acknowledgment that not everything can be covered.  We know that through our own experience with insurance…but rarely do we ask the public at large to help articulate more specifically what that means.

Could policymakers engage in a priority-setting process as well as the public itself? Absolutely….many people have suggested they do so.

But the ACA presents a unique opportunity to ask the public to participate in a way that few have done: to help guide how trade-offs can be made in the fairest way possible.

The future users of the EHB are the ones we need to learn from. And that is what deliberation really is: the opportunity for policymakers to learn how their constituents respond when there are no easy answers.

Slide 26

Questions, comments?
Kristin L. Carman
(202) 403-5090

Jessica Waddell
(202) 403-5947

Slide 27

Community Forum deliberative methods team


  • Kristin L. Carman, PhD, PD
  • Jess Fernandez
  • Steve Garfinkel, PhD
  • Dierdre Gilmore, MA
  • Susan Heil, PhD
  • Diane Martinez, MPH
  • Maureen Maurer, MPH
  • Alex Ortiz
  • Karthik Shyam, MPP
  • Kip Thomson
  • Jessica Waddell, MPH
  • Amy Windham, PhD
  • Marilyn Moon, PhD, CF PI


  • Center for Healthcare Decisions
  • Ela Pathak-Sen
  • Marthe Gold, MD
  • Shoshanna Sofaer, DrPH
  • Stanford University, Center for Deliberative Democracy
  • Stanford University, Symbolic Systems Program


  • Joanna Siegel, ScD

Slide 28


  • Fishkin, J. S. (2009). When the people speak: Deliberative democracy and public consultation. Oxford University Press.
  • Gracia, D. (2003). Ethical case deliberation and decision making. Med Health Care Philos,6(3), 227–233.
  • Gutmann, A., & Thompson, D. (2004). Why deliberative democracy? Princeton, NJ: Princeton University Press.
  • Kim, S. Y., Uhlmann, R. A., Appelbaum, P. S., Knopman, D. S., Kim, H. M., Damschroder, L. et al. (2010). Deliberative assessment of surrogate consent in dementia research. Alzheimer’s Dement, 6(4), 342–350.
  • Matthews, D. (1994). Politics for people: finding a responsible public voice. Chicago: University of Illinois Press. P110.
  • Pesce, J.E., Kpaduwa, C.S., Danis, M. (2011). Deliberation to enhance awareness of and prioritize socioeconomic interventions for health. Soc Sci Med; 72: 789-797.