Skip Navigation
Department of Health and Human Services

Back to "About This Webinar"

Using Deliberative Methods to Engage Patients, Consumers, and the Public

Webinar Transcript, December 6, 2011

Slide 1

Using Deliberative Methods to Engage Patients, Consumers, and the Public
Agency for Healthcare Research and Quality
Community Forum
December 6, 2011

Speaker notes: Welcome, everyone, to this webinar about using deliberative methods to engage patients, consumers, and the public

Slide 2

Purpose and objectives

  • Purpose
    • Describe deliberative methods
    • Explore how deliberative methods can be used in your work
  • At the end of this, you should be able to:
    • Identify situations appropriate for deliberative methods
    • Develop the questions that can be addressed through deliberative methods
    • Understand how deliberative methods can be used

Speaker Notes: First, we want to go over the purpose of today’s session.  We will describe deliberative methods – providing key characteristics of deliberative methods and helping you to recognize when deliberative methods are appropriate to use.  We will also explore how deliberative methods can be used in the work that you or your organization does to reach patients, consumers, caregivers, and lay members of the public.

At the end of this session, you should be able to:

  • Identify situations that warrant deliberative methods
  • Develop questions that can be addressed through deliberative methods
  • Understand how deliberative methods can be used

Slide 3

Presenter introductions

  • Kristin L. Carman, PhD, Managing Director, American Institutes for Research (AIR)
  • Ela Pathak-Sen, Director, Commotion
  • Jessica Waddell, MPH, Researcher, AIR
  • Marge Ginsburg, MPH, Director, Center for Healthcare Decisions

Images: Each presenter has a corresponding photograph 
Speaker notes: Now, I want to introduce myself and my colleagues who will talk with you today.

I am Kristin Carman, Managing Director at the American Institutes for Research.  Thank you all for joining.  We’re really excited that you are participating.  We have 253 people on the line, coming from the Food and Drug Administration, Fraser Health, Institute of Medicine, National Health Council, National Partnership for Women and Families, Ovarian Cancer Advocacy Alliance, Pacific Business Group on Health, and the University of North Carolina, among many others.

I want to introduce my co-presenters: Ela Pathak-Sen, Jessica Waddell, and Marge Ginsburg.  I also want to recognize Jess Fernandez, our research assistant, who has helped us tremendously in our preparation.

Also, Joanna Siegel, our Project Office at AHRQ, is on the line and has contributed a great deal to this presentation.

Slide 4

Reason for this work

  • Agency for Healthcare Research and Quality’s 3-year initiative called Community Forum
  • Led by the American Institutes for Research (AIR) with key partners
  • Major part of this project is to:

Advance the use of deliberative methods for obtaining input from members of the public on a health research topic
Speaker notes: This webinar is part of a large, three-year project, funded by the Agency for Healthcare Research and Quality (AHRQ) and led by the American Institutes for Research (AIR).

Initiated in August 2010, this project entails two key parts:

    • To expand stakeholder engagement in AHRQ’s Effective Health Care Program,
    • To advance the use of public deliberation or deliberative methods for obtaining lay public input a health research topic.

This second part of the project seeks to expand the evidence base about public deliberation and garner public input on a health topic related to comparative effectiveness research to inform AHRQ’s EHC Program.  To do so, we are conducting a randomized controlled experiment comparing five deliberative methods to each other and to a control group.  This webinar focuses on the second part – regarding what we have learned about deliberative methods to date.

The basis for this presentation is an extensive literature review of public deliberation, including international applications and applications in fields such as health, bioethics, education, politics, and environmental policy, conducted in fall 2010 – early 2011.   The literature review included peer-reviewed and grey literature, and key informant interviews.

Further, as part of the randomized control trial (RCT) we will be conducting, we have undergone an extensive planning process over the past year, informed by the literature review, input from our Technical Expert Panel, and AHRQ, and gleaned lessons that are immediately applicable to others’ work in deliberative methods, which we share today. 

Slide 5


  • Overview of deliberative methods
  • Application to health
  • Examples from the literature
  • IOM committee report on essential benefits
  • Interactive exercise
  • Discussion and Q & A

Speaker notes: As a reminder, we will discuss, first, an overview of deliberative methods – covering defining features of deliberation, the issues that are appropriate for deliberative methods, and the best practices in designing deliberative methods.

Next, we will highlight the application of deliberative methods to health through some key examples. 

Then, we will present a few real-world applications of deliberative methods in health.

Finally, we will move into an interactive exercise around starting a deliberative process.  Using some of the issues your organizations shared with us prior to the webinar, we will discuss how deliberative methods can be used to address these issues.

Slide 6

Overview of Deliberation
Speaker notes: First, we will start with an overview of deliberative methods.

Slide 7

What is deliberation

  • Convening of “mini publics” (Fishkin 2009) …
  • …“to weigh carefully the consequences of various options for action and the views of others” (Matthews 1994)
  • Provides opportunity to weigh the “principles and values involved as well as the circumstances and consequences” of the topic of debate (Gracia 2003)

Speaker notes: So, what is deliberation?  Public deliberation is founded on deliberative democratic theory.  It entails, first, bringing lay members of the public together – in “mini publics” (small groups of people who can represent the broader public). Deliberative methods – which are distinct applications of public deliberation – can include those individuals who are disproportionately or highly affected by an issue such as patients.  If you include individuals who are particularly affected by an issue, you need to ask them to consider their views in relation to everyone else – to encourage a societal view.

Together, they learn about and discuss an issue – this has been referred to as the careful weighing of the sides of an issue, considering the various principles and values involved along with the consequences of any action to society overall.

Deliberative democratic theory specifies that such interaction results in informed public opinion, eliciting perspectives of individuals once they have had the opportunity to carefully think about an issue.  The key is that such a process elicits how people actually think and feel about an issue once they learn more about it and hear from others’ perspectives and that this resulting opinion is more stable and representative of how the public feels than opinions that would be garnered through more traditional forms of consultation such as opinion polling.

The goal of deliberation is fundamentally to encourage people to learn and think openly about an issue, hearing others’ perspectives, so that in the end, conveners or sponsors of deliberation learn from the participants.

Slide 8

Goals of deliberative methods

  •  To provide decision makers with understanding of public values relevant to complex issues
  • To influence change at policy or program level
  • To expand participants’ knowledge and insight on an issue
  • To increase participants’ civic engagement and willingness to participate

Speaker notes: Next, what are the goals for deliberation?  Why would you do this?  Deliberative methods can provide decision-makers with an understanding of public values relevant to complex issues.  One way it does this is by including diverse perspectives, including those who may be disproportionately affected by an issue or who are traditionally excluded from such processes.

It can influence change at a policy or program level.  Having the public’s input can be powerful vis-à-vis working with other stakeholders, which may motivate change.   There is some – albeit limited – evidence that public deliberation affects policy or program level changes.  The key here is the early engagement of and delivery of the participants’ input to decision-makers.

Further, deliberation can expand participants’ knowledge and insight on an issue.  Through the educational process, deliberative processes can reveal information gaps, misinformation, and misunderstandings—which are addressed through the deliberative process. Based on evaluations of public deliberation found in the literature review, deliberation was found to impact participants’ knowledge and attitudes on an issue.  For example, participants may demonstrate that they gained additional knowledge about the topic and their attitudes may change.

The literature review also revealed that previous evaluations have found that deliberation increases participants’ civic engagement and willingness to participate in civic activities in the future.

Slide 9

Where deliberative methods fit

  • Intensity of involvement
  • Image: Arrow pointing to the right with three boxes on top of it. Each box from left to right has words in it: “Opinion polling,” “Focus groups,” “Deliberative methods,” respectively.
  • Informed input

Speaker notes: We wanted to give you an idea of where deliberation fits in with other public consultation methods—how does it compare in terms of intensity of involvement and the resulting input from the participants. 

While there are many other activities along this spectrum and arguably more intensive activities that follow deliberative methods, this diagram is intended to give you a high-level understanding of where deliberation fits in. 

Generally speaking, deliberative methods are approaches to public consultation or political participation that are more intensive than more traditional approaches such as opinion polling or focus groups.

Opinion polling—on the far left—is a common way in which the public gives feedback or provides input.  However, opinion polling elicits top-of-mind responses – deliberation, alternatively, elicits more stable, considered input because the process iteratively involves education and conversation with diverse individuals, enabling individuals to reevaluate information and their positions.

Moving to the right, deliberative methods—just like focus groups—involve the convening of participants in group discussions.  However, deliberative methods are more intense than traditional focus groups, as we will highlight in the next slide.

Slide 10

Components of deliberative methods

  • Image: Arrow pointing to the right with three boxes. Each box from left to right has words in it: “Opinion polling,” “Focus groups,” “Deliberative methods,” respectively.
  • Convening of groups
    • Discussion
      • Education
      • Reason-giving and debate
      • Societal perspective

Speaker notes: Both focus groups and deliberative methods involve the convening of groups and discussion.  However, deliberation is more intensive because: (1) it consists of an educational component to ensure that all participants are starting from a shared level of knowledge and understanding, (2) emphasizes a reason-based dialogue and debate, and (3) aims to elicit a community-based perspective and thus orient the conversation to societal-level thinking. 

Further, education is a defining feature of deliberation, as I mentioned previously. 

Additionally, because a key goal of deliberation is to encourage participants to hear from others on the issue, deliberation involves active debate. The idea is that each participant’s points of view will be expanded through this process.  It is also in the debate that the various values and ethics involved will emerge from the participants’ perspectives. 

Another element of deliberation is reason-giving – which is the act of providing the rationale for your statements.  This may be a new exercise for many participants, but it is essential to deliberative democratic theory.  Requiring that participants share their reasoning not only enriches their input but also enables other participants to more fully respond to their comments.  It also instills accountability – people are asked to explain their reasoning, not just make statements. 

Finally, deliberative methods involve asking participants to assume a societal perspective (sometimes, as societal decision-makers).  The idea here is that deliberation is intended to provide insight into how people feel about the issue when they consider everyone and the possible consequences that will affect society.

Slide 11

Characteristics of deliberative topics

  • Values-based or ethical dilemmas
  • Social/affecting common good
  • Cannot be resolved through technical or scientific information alone
  • Controversial but opportunity for common ground
  • Timely and relevant

Speaker notes: Here, we highlight the criteria for selecting topics for deliberation.

Topics appropriate for deliberative methods consist of values-based or ethical dilemmas—often where values are potentially in conflict or various ethical/moral issues need to be balanced and reconciled.  An example issue may be the decision between incenting public transportation vs. deterring personal transportation; this issue involves concerns for environment, sustainability of public transportation system, equity considerations (e.g., do all communities have access to public transportation?), traffic considerations, toll on infrastructure (i.e., more cars=more toll), personal convenience, and the impact on the auto industry, among other considerations.

Topics appropriate for deliberation are also social issues that affect the common good or the community – there needs to be a stake for a member of the public.  For example, building on the same issue – there is a stake for everyone in environmental wellbeing and the availability of public transportation, so this issue is appropriate for deliberation.

Further, appropriate topics or issues cannot be resolved through technical or scientific information alone.  The key is that members of the public, patients, or consumers are engaged for their “life expertise” – their perspectives and experiences are important for resolving or shedding light on the issue.  Extending the same example further – this issue requires consideration of multiple stakeholders' perspectives, most importantly the public's.  An issue such as inspecting buses for safety is not a deliberative topic because there is a technical answer.  The key is whether other perspectives and multiple pieces of information – such as values – are relevant.

Topics should also be controversial but provide opportunity for common ground – for example, retirement benefits.  People may have deeply held, emotionally charged views on this issue about responsibility, entitlement, method of financing, etc. of retirement benefits; however, despite these strong feelings, there is an opportunity to come together because society has a stake in how well people are prepared for retirement.  Issues such as abortion in the U.S. may not be best addressed through deliberation because the feelings are so deeply entrenched that movement toward a shared goal may not be feasible.

Finally, issues should be timely and relevant to current public policy or decisions so that participant input will be used to affect policy, programs, or other decision-making.  For example, deliberating on the War in Iraq would not be timely now – but discussing troop withdrawal, Afghanistan, Pakistan, and/or Iran is relevant to current policy and decision making.  You could still deliberate on an issue about which policymakers or other decision-makers have made a decision but you think a public voice could counter the decision or be used in future decisions if the issue is a persistent societal one (e.g., how we finance public education, how we subsidize agriculture).

Slide 12

How deliberative methods vary

  • Length, duration
  • Group size, participant sample
  • Recruitment method
  • Structure (e.g., breakout groups, interrupted)
  • Education, use of experts
  • Mode (online, in-person)
  • Facilitation
  • Consensus as goal

Speaker notes: There are many distinct deliberative methods with defining features in the field, including citizens’ juries, consensus conferences, and Deliberative Polling, among others, and many examples of hybrid methods – which borrow key features from the distinct methods.  Because a discussion of the methods can be laborious, we wanted to, instead, give you a sense of how deliberative methods vary.

They can vary in length and duration – with ranges from 2 hours to a standing commitment of 3 years (with intermittent meetings during this period).

They can also vary by recruitment method – some methods use convenience sampling; some use random sampling.  This depends on whether a representative sample of the lay public is a goal.

Methods also vary by group size – ranging from groups of about 12 to upwards of 500. 

You would select among these features based on resources, goals, desire for representativeness, and nature of topic (for instance, highly emotionally or politically charged issues may be better for an interrupted structure – in which participants could have a break, whereas a complex, technical issue may warrant the use of multiple experts). 

Now, I will transition to Ela, who will tell you more about how to best design and implement these methods.

Slide 13

Steps to designing deliberative methods

  • Identify and define issue needing input
  • Engage the decision makers for whom results are intended
  • Set goals for deliberation
  • Determine participant population (e.g., lay public, patient group)
  • Select deliberative process or create hybrid
  • Determine evaluation approaches

Speaker notes: There are a few key steps to designing deliberative methods including:

  • First, identifying and defining the issue needing input – here is where you think about the issues that you want to bring to your constituent groups or community members. 
  • Second, you should consider involving policy and decision makers in the process.  You could engage them early in the process – during the formative stages – or consider presenting/sharing the findings from the deliberation with them to see how they would use the input.  Early engagement is ideal.
  • Third, you should set goals for the deliberation.  This often entails clarifying how you think your target participant group could make a difference, what information you would like to obtain, and what intended outcomes for the engagement you would like to see.
  • Next, you want to consider your target participant population – who do you want to engage. 
  • Then, you should select the deliberative method or develop a method.  Key features of methods include the length, mode (in person or online), group size, etc.  Key features of methods include the length, mode (in person or online), group size, as I just mentioned.  When you select a method, you are also making decisions about the nature of the educational component, style of facilitation (e.g., active/passive), and whether you will use experts.
  • Finally, you need to establish any evaluation approaches, such as pre/post surveys.  These may include assessing the fidelity of the implementation to the method.

Slide 14

Best practices

  • Early involvement of audience for participant input
  • Diversity
  • Balance
  • Transparency
  • Clarity
  • Adapt to context

Speaker notes: Based on the literature review, best practices in designing deliberative methods include, first the early involvement of the audience for the public’s input, as we have discussed.  This is key for both assuring participants that their input will be used and valued and also for facilitating the uptake and use of the participants’ input.  This is especially important when deliberation may be a new method to policy or decision makers.

Further, diversity is key.  It is important to consider diversity in all aspects of the design – including the participant recruitment, group composition, educational materials and modes (e.g., written, audio/video), languages used and available, experts’ identities and perspectives, and facilitators, among other considerations.  Since diversity is essential for promoting one of the goals of deliberation, i.e., rich debate, designing deliberative methods so that they include diversity of perspectives is crucial for promoting good outcomes.

Balance in the perspectives presented is also important.  This means that multiple sides of the issue are presented and given their due time.  For instance, regarding an issue such as mandatory vaccination, you would want to include such experts as public health practitioners, pediatricians, environmental health experts, ethicists, and proponents of individual liberties.  The key is to make sure that the educational process, which may include experts, is fair to each side of the issue.  This component not only bolsters the richness of the debate but also promotes the transparency and legitimacy of the process.

A fourth best practice is transparency of process.  This entails sharing the process with participants, policy and decision makers, and other external observers.  You will want to make sure that participants understand why they are being involved, what their role is and what is being asked of them, how their input will be used (if known), and what follow up with them will be like, if there will be any at all.  In addition, you will want to make sure that your audience for the public’s input understands the educational process and how input was elicited.

Clarity is also important.  As highlighted when discussing transparency, it is important to be clear with participants regarding their role and what they will be asked to do.

Finally, it is important to adapt your methods, materials, and overall approach to the current political, social, and cultural contexts.  Essentially, keeping an eye on the environmental factors (e.g., media coverage, an election) that may influence how participants feel about an issue is important for both designing and implementing your method as well as any evaluation of it.

Slide 15

Applications of deliberative methods to health care

Speaker notes: Now, we will consider how deliberative methods are relevant and applicable to healthcare.

Slide 16

How deliberative methods can help inform health care and health research

Example issues How deliberation can help

Priority setting comparative effectiveness research topics

Balancing of values, prioritization of topics

Organ shortages for transplantation

Guidance on societal views of ethics involved

Racial/ethnic disparities in maternal  and infant mortality

Propose solutions and establish goals for improvement

Quarantine during public health emergencies

Balancing of societal wellbeing vs. individual rights

Speaker notes: Here, we have listed a few example issues in public health or healthcare that are applicable to public deliberation on the left.  And on the right hand side, we have listed how deliberation could help. 

For example, priority-setting for comparative effectiveness research.  The American Recovery and Reinvestment Act allocated substantial funding to the conduct of CER and development of research infrastructure to support this important research in the future.  There are many conditions, treatments, and subpopulations that warrant CER studies; however, with limited research resources and time, using a deliberative process can reveal the public’s values in terms of research and which conditions, treatments, or populations they feel should be prioritized – and why.  Ultimately, deliberation on this topic could result in a prioritization of topics.

Another example is the ethical dilemma posed by organ shortages for transplantation. In the literature, we found several examples of public deliberation being used for bioethical topics such as this one.  Deliberation could provide guidance on societal views on the ethics involved and how they should be balanced.

Deliberation could also be used to address health disparities issues such as racial/ethnic disparities in maternal and infant mortality.  The literature included some examples of deliberation in which participants were asked to consider a health problem and develop ideas for solutions or goals for addressing it.  Deliberation on such topics raises awareness and can help generate ideas that can inform public health policy or programs.

Public health inherently involves dilemmas such as the fundamental tension between societal wellbeing and individual rights.  Because of this, deliberation can be used in emergency preparedness.  Several examples of deliberation in the literature involved public health issues.  Input on these issues can inform public health planning.

Slide 17

Real-world applications of deliberative methods

  • Two examples from the literature
    • Surrogate consent for research participation
    • Priority-setting social and health interventions
  • IOM’s Committee on Determination of Essential Health Benefits, Oct. 2011

Speaker notes: To make all of this explication more concrete, we are going to highlight several key examples of deliberation – two of which come from the literature.  The first addresses surrogate consent for research participation. 
The second study asked participants to prioritize social and health interventions based on the interventions they believed would have the biggest impact on health.
Finally, Marge Ginsburg will share some of her experience and insights from serving on the Institute of Medicine’s Committee on the Determination of Essential Health Benefits, who recommended that public deliberation be used in determining the benefits.

Slide 18

Examples from the literature

Speaker notes: Now, we will cover some examples from the literature review.

Slide 19

Deliberative method for assessing surrogate consent

  • Issue: Can family members provide surrogate consent for research participation for individuals with dementia?
  • Researchers: led by University of Michigan with funding from National Institute on Aging
  • Goal of deliberation:
    • Assess how participants view surrogate consent for research participation for individuals with dementia
    • Evaluation:  Determine whether deliberation affected caregivers’ views of surrogate consent

Speaker notes : The first study we will highlight entailed a deliberative method for assessing surrogate consent policies.  This study sought to address the issue of surrogate consent for research participation and determine the effect of deliberation on people’s views of surrogate consent. As Kristin described, it is key that issues for deliberation are complex and cannot be resolved through technical information alone.  In the U.S., regulations allow research with incapacitated adults based on consent by their legally authorized representatives. However, states have tended not to define what a legally authorized representative is.  Further, there is a lack of consensus rega