Skip Navigation
Department of Health and Human Services www.hhs.gov
  • Home
  • Search for Research Summaries, Reviews, and Reports
 
 

EHC Component

  • DEcIDE Project

Topic Title

  • Options for Developing a Web-based Forum on Patient Registries
Save this page in Facebook.com  Save this page in Myspace.com  Save this page in Twitter.com  Save this page on your Google Home Page  Save this page in Windows Live
Save this page in Yahoo  Save this page in Ask.com  Stumble this page.  Save this page in del.ico.us  Digg this page. 

E-mail E-mail   Print Print

Add to My Collections



Research Report - Final – Jun. 2, 2014

Options for Developing a Web-based Forum on Patient Registries

Formats

Table of Contents

Authors and Affiliations

Richard E. Gliklich, M.D.
Daniel M. Campion, M.B.A.
Laura Khurana, M.P.H.
Michelle B. Leavy, M.P.H.
Jannette Karl, M.B.A., P.M.P.

The authors are affiliated with the Outcome DEcIDE Center, Cambridge, Massachusetts.

None of the authors have any affiliations or financial involvement that conflicts with the materials presented in this report.

Acknowledgements

The authors would like to thank all those who contributed to this project, including the speakers and participants at the stakeholder meeting and the individuals who shared their experiences with Web-based communities of practice. All of the contributors to this project participated as individuals and not necessarily as representatives of their organizations. No statement in this document should be construed as an official position of any organization represented here.

Structured Abstract

Background

Patient registries are an important tool for collecting observational, real-world clinical data. There is a need for current information on best practices for planning, operating, evaluating, and analyzing data from patient registries, as well as information on other practical issues in registry science. A Web-based, collaborative forum is a potential way to meet the information needs of registry developers and users. The objectives of this project are to determine the potential value of a Web-based collaborative forum, identify stakeholders’ needs for such a forum, and propose possible strategies to create a forum.

Methods

Information for this project was collected through background research and stakeholder engagement activities. Stakeholder perspectives were gathered through an in-person stakeholder meeting, held in March 2012.

Results

Stakeholders clearly articulated the consensus that a Web-based CoP is needed and would be welcomed in the domain of patient registries. They agreed that the added value of such a Web-based CoP would primarily be determined by the extent to which it was relevant to their work and helped them to do their work better, faster, or more easily. Secondary goals mentioned by stakeholders included facilitating networking, interaction, and collaboration; improving efficiencies in resource use and reducing duplication of effort; improving patient care and outcomes; providing an organized system for learning and information sharing; and serving as a collective voice for registries. Stakeholders also suggested requirements for the governance, management, and technical features of the CoP.

Conclusions

The background research and the input from stakeholders suggest that a Web-based CoP is a feasible, practical way to provide current information on patient registry best practices and methods to a diverse set of stakeholders.

Executive Summary

Patient registries are an important tool for collecting observational, real-world clinical data, and they are increasingly used by a variety of stakeholders for a broad range of purposes. This growing interest has created a need for current information on best practices for planning, operating, evaluating, and analyzing data from patient registries, as well as information on other practical issues in registry science. A Web-based, collaborative forum is a potential way to meet the information needs of registry developers and users.

The objectives of this project are to determine the potential value of a Web-based collaborative forum, identify stakeholders’ needs for such a forum, and propose possible strategies to create a forum. A key component of this project is engagement with stakeholders, including Federal partners, funding agencies, industry sponsors, researchers, health care providers, payers, and patients, to ensure that their views are considered and incorporated. The goal of this paper is to provide actionable information to the Agency for Healthcare Research and Quality (AHRQ) for developing a Web-based collaborative forum, should it be determined that such a forum is both feasible and valuable.

Information for this project was collected through background research and stakeholder engagement activities. Literature reviews and Internet searches were conducted to define the concepts of “Web-based, collaborative forum” and “community of practice (CoP),” and to identify and characterize existing examples of Web-based forums. Stakeholder perspectives were gathered through an in-person stakeholder meeting, held in March 2012. More than 70 stakeholders attended the meeting, where invited speakers from existing Web-based forums and CoPs discussed the concept of CoPs and lessons learned from their experience managing CoPs and Web-based forums. Stakeholders shared their perspectives on the need for a Web-based forum on patient registries and described the features that they would like to see in such a forum.

Stakeholders clearly articulated the consensus that a Web-based CoP is needed and would be welcomed in the domain of patient registries. They agreed that the added value of such a Web-based CoP would primarily be determined by the extent to which it was relevant to their work and helped them to do their work better, faster, or more easily. Secondary goals mentioned by stakeholders included facilitating networking, interaction, and collaboration; improving efficiencies in resource use and reducing duplication of effort; improving patient care and outcomes; providing an organized system for learning and information sharing; and serving as a collective voice for registries. Stakeholders also suggested requirements for the governance, management, and technical features of the CoP.

The background research and the input from stakeholders suggest that a Web-based CoP is a feasible, practical way to provide current information on patient registry best practices and methods to a diverse set of stakeholders. The Web-based CoP should be designed to meet the primary and secondary goals as stated above and should be hosted on an independent Web site, using an off-the-shelf technology platform, of which there are many options. The Web site should be able to facilitate discussion forums and support a resources section, member directory, Webinars, and teleconferences. The sponsor of the Web-based CoP should be one or more entities that are perceived as unbiased; stakeholders mentioned AHRQ or a public-private partnership as possible sponsorship models. The CoP should be governed by an advisory group or steering committee comprised of representatives from multiple relevant registry stakeholder groups. A charter and communication plan should be written, and plans for sustained funding should be outlined. Membership for the Web-based CoP should be broadly open and not ES-1 restricted by factors such as geographic location or experience level. The topical direction and marketing plan for the CoP should be informed by member input.

Launching and maintaining a Web-based CoP on patient registries is feasible from a technical and operational standpoint, as is demonstrated by the many examples of Web-based CoPs in other domains. Cost drivers include the type of sponsor (a Federal agency sponsor introduces additional administrative and regulatory requirements), the scope of the CoP (including the range of topics covered), and the level of active moderation and content generation conducted by CoP staff. Immediate next steps for AHRQ, should it decide to establish a Web-based CoP on patient registries, include determining the scope and target audience of the Web-based CoP; determining what internal and external resources are available for this initiative; establishing a stakeholder steering committee; and determining the functional requirements for the technology platform.

Introduction

Patient registries are receiving increased attention as an important tool for collecting observational, real-world data to fulfill multiple purposes. Broadly, a patient registry is “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes.”1

Registries are used by a variety of stakeholders for a broad range of purposes. Clinicians value the ability of registries to rapidly collect data about disease presentation and outcomes from large numbers of patients, producing a real-world picture of disease, current treatment practices, and outcomes. Physician organizations use registry data to benchmark physicians’ performance against evidence-based guidelines, focus attention on specific aspects of a particular disease that might otherwise be overlooked, or provide data for clinicians to compare themselves with their peers. For a payer, registries can provide detailed information from large numbers of patients, including how procedures, devices, or pharmaceuticals are actually used and on their effectiveness in different populations, which could be useful in determining coverage policies. For a drug or device manufacturer, a registry-based study might demonstrate the performance of a product in the real world or meet a post-marketing commitment for safety surveillance.2

Government agencies and other funding sources also are increasingly interested in patient registries as a component of evidence-based medicine. In 2010, Congress authorized the formation of the Patient-Centered Outcomes Research Institute (PCORI) “to conduct research to provide information about the best available evidence to help patients and their health care providers make more informed decisions.”3 In December 2012, PCORI published Methodology Standards that highlight patient registries as a potential source of evidence for patient-centered outcomes research.4 The Agency for Healthcare Research and Quality (AHRQ) has also made significant investments to further the science of patient registries. AHRQ has funded the development of “Registries for Evaluating Patient Outcomes: A User’s Guide,” which provides comprehensive information on planning, designing, operating, analyzing, and evaluating patient registries. The document was first published in 2007, with a second edition published in 20101 and a third edition released in April 2014. In addition, AHRQ has funded the development of the Registry of Patient Registries (RoPR), a searchable database of existing patient registries in the United States.5 Through its Effective Health Care Program and Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network, AHRQ has a history of promoting the development of registry methods.

Rationale

The increased interest in and use of registries has led to a growing need for current information on best practices in registry methods and science relevant to the broad range of stakeholders. Print and online resources, such as “Registries for Evaluating Patient Outcomes: A User’s Guide,” provide valuable information on these issues, but they are typically static documents and are invariably limited in breadth of topics and speed of updates. In addition, many emerging areas of registry science are too new to be summarized in a formal document but still merit considerable debate and discussion. A Web-based, collaborative forum is a potential way to meet the growing information needs of registry developers and users. The primary purpose of such a forum would be to facilitate the exchange of information among persons interested in learning more about patient registries. An open, collaborative forum could thus 1 complement more carefully vetted publications and would also be a natural dissemination pathway for registry-related work or findings produced by AHRQ and other Federal partners. The idea of a “Web-based collaborative forum” reflects the broader concept of a “community of practice” (CoP), as discussed in the Background Research section below. Such a forum for patient registries could also be an important next step in AHRQ’s evolving strategy to advance the science and use of patient registries and to fulfill AHRQ’s broader mission of improving the quality, safety, efficiency, and effectiveness of health care for all Americans.

Project Objectives

The objectives of this project are to determine the potential value of a Web-based collaborative forum, identify stakeholders’ needs for such a forum, and propose possible strategies to create a forum. A key component of this project is engagement with stakeholders, including Federal partners, funding agencies, industry sponsors, researchers, health care providers, payers, and patients, to ensure that their views are considered and incorporated. The goal of this paper is to provide actionable information to AHRQ for developing a Web-based collaborative forum (should it be determined that such a forum will be both feasible and valuable) that will be relevant to the needs of the Medicare, Medicaid, and other Federal health care programs and will reflect the overall goals of the Effective Health Care Program.

This report begins by describing the background research and stakeholder engagement activities that were conducted as part of this project. The report then summarizes the findings from these activities and presents recommendations for the goals and objectives, technical features, governance, and management of a Web-based forum. The report concludes by discussing the feasibility of such a forum and proposing next steps for AHRQ. Appendix A contains a glossary of technical terms used throughout this report.

Methods

Information for this project was gathered through literature reviews, Internet searches, discussions, and a large in-person stakeholder meeting. Literature reviews and Internet searches focused on two areas: (1) defining the concepts of "Web-based, collaborative forum" and "community of practice," and (2) identifying existing examples of Web-based CoPs or forums that serve various audiences and are managed by different types of sponsors. Relevant information was located through electronic searches of PubMed, Google, Google Scholar, and ClinicalTrials.gov, as well as review of public information, such as Web sites and press releases. Information about existing CoPs and forums was collected to understand their purpose, target audience, key features and services, and infrastructure (technology, human resources, funding). This information is summarized in the Background Research section of this report.

In the course of this background research, three individuals with experience in initiating and managing Web-based CoPs or forums in both health care and non–health-care-related fields were identified. Following continued and in-depth discussions with these individuals, the project team invited them to speak on their experiences at the in-person stakeholder meeting held for this project. Their experience is summarized below.

  • Joanne Cashman, Ed.D., is project director for the Individual with Disabilities Education Act (IDEA) Partnership, sponsored by the Office of Special Education Programs in the U.S. Department of Education. The IDEA Partnership aims to create opportunities for partner organizations to work across Federal agencies, Federal investments, national organizations, State agencies, and stakeholder groups to build capacity of States, districts, and schools to improve results for students with disabilities and to learn to share the implementation of IDEA.
  • Margaret Farrell, M.P.H., R.D., is communications specialist at the National Cancer Institute (NCI), National Institutes of Health (NIH). Ms. Farrell provides programmatic leadership for Research to Reality, NCI's online CoP for cancer control researchers and practitioners.
  • Mamie Jennings Mabery, M.A., M.Ln., is acting director of the Knowledge Management Office within the Office for State, Tribal, Local and Territorial Support at the Centers for Disease Control and Prevention (CDC) in Atlanta, GA. Ms. Mabery launched a CoP program in 2008 to foster collaboration across the health system. As manager of that program, Mamie and her team created an online CoP Resource Kit6 for learning about and launching CoPs and co-created phConnect, a Web-based collaboration platform that now supports more than 4,000 public health professionals in more than 120 active communities.

The in-person stakeholder meeting was held on March 26, 2012, in Arlington, VA. Information about the meeting was disseminated through a public announcement on the AHRQ Web site and a general email announcement sent to more than 300 individuals who had participated in stakeholder meetings for the RoPR project. Individuals who responded to the announcement were registered for the meeting on a first-come basis, but invitations for some categories of stakeholders with a strong response (e.g., researchers) were capped and waiting lists were created to ensure equal representation of all stakeholder groups.

A total of 73 stakeholders attended the meeting. Participants represented a variety of communities, including researchers (e.g., academia, American Institutes for Research), health care providers and provider associations (e.g., American Medical Association, American College of Rheumatology, American Academy of Family Physicians), government (e.g., NIH, Food and Drug Administration, CDC), industry (e.g., software and pharmaceutical companies), payers (e.g., Centers for Medicare and Medicaid Services, Humana, America's Health Insurance Plans), and patient representatives (e.g., Cystic Fibrosis Foundation, National Psoriasis Foundation, National Foundation for Celiac Awareness), as displayed in Figure 1. A full list of organizations represented at the meeting is included as Appendix B.

Figure 1. Meeting participants by stakeholder group (N=73)

Figure 1 is a pie chart divided into six stakeholder groups, depicting the number of participants from each group that attended the stakeholder meeting in March 2012.  There were 20 researchers, 13 healthcare providers, 13 government representatives, 10 industry representatives, 9 payers, and 8 patient representatives, for a total of 73 participants.

The in-person meeting combined presentations from guest speakers who explained the concept of a Web-based forum and provided real-world examples of forums and CoPs with open discussion and feedback from stakeholders. The agenda from this meeting is included as Appendix C. Stakeholders were seated in groups according to stakeholder type and were encouraged to designate an individual to take notes during the small group discussion portions of the meeting. These notes were returned to project team after the meeting. During the large group discussions, the project team took notes on large flipcharts to document the ideas expressed and any consensus reached. Together with the audio recording of the meeting and the background research conducted previously, these notes were then used to inform the findings and recommendations presented in this report.

At both the in-person meeting and during individual discussions with stakeholders, the project team focused on understanding stakeholder needs and perspectives. To explore the rationale for a potential Web-based CoP on patient registries, stakeholders were asked to describe the current knowledge-sharing environment around patient registries and whether they perceived a need for a Web-based CoP within this environment. Having found that a need for a CoP existed, stakeholders then discussed the value that they would draw from a Web-based CoP, and how they saw such a tool advancing the science and practice of patient registries. Stakeholders also identified the specific features and characteristics of a Web-based CoP that were of highest priority and would confer the most value. Finally, stakeholders suggested some questions and issues that the sponsor of a Web-based CoP would need to address, but on which the stakeholders themselves did not reach a clear consensus. Stakeholder perspectives are summarized in the Findings section of this report.

Background Research

Communities of Practice

As noted above, the idea of a “Web-based collaborative forum” expressed in this task order reflects the broader concept of a CoP. CoPs are formed by people interested in particular domains of human endeavor who seek to learn more from each other through a process of ongoing interaction. Etienne Wenger, an educational theorist and practitioner, offers the following definition, “Communities of practice are groups of people who share a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis.”7 Wenger and anthropologist Jean Lave coined the term while studying apprenticeship as a learning model to refer to the complex set of social relationships that act as a living curriculum for an apprentice. Sometimes referred to as learning networks, thematic groups, or tech clubs, CoPs can be found in many environments, including business, education, public health, professional associations, and international development.

In the area of health care practice and research, professional associations often serve as de facto CoPs for practitioners in a specific clinical field. Practice-based research networks (PBRNs) can also serve this role for primary care providers; AHRQ defines PBRNs as “groups of primary care clinicians and practices working together to answer community-based health care questions and translate research findings into practice.”8

CoPs share three primary characteristics:

  • The domain is a topic, issue, or concern of interest shared by the individuals who participate in the community. Participation implies a commitment to the domain and a shared competence that distinguishes members from other people.
  • The community forms as members share information and engage in discussions and other helpful activities in pursuing their interest in the domain. These relationships enable them to learn from one another. As such, a Web site in itself would not be considered a CoP. Rather, the individuals who use the Web site and perhaps other mechanisms (e.g., in-person meetings) to facilitate learning would be considered a CoP.
  • The practice that members develop together refers to the common repertoire of resources they discover and use over time, including experiences, stories, tools, and ways of addressing recurring problems. Forming these collected resources takes time and sustained interaction.

Communities develop their practice by responding to the practical needs of members. These activities commonly take the form of problem solving; articulating and responding to requests for information; seeking out the experiences of one another; reusing assets (e.g., proposals, letters, formulas); discussing developments (e.g., new technologies, regulations, research findings); documenting developments (e.g., case studies, data collection); conducting site visits; mapping knowledge and identifying gaps (e.g., research agenda setting); and coordinating and conducting special projects.

Four key processes are central to fostering a learning environment for CoP members, as illustrated in Figure 2. Diverse members “bring the practice in” by describing their experience and sharing stories. Members also “push the practice forward” by encouraging the examination and development of their practice, questioning assumptions, and exploring new ideas. The CoP helps members “create self-representation” by deriving lessons and finding ways to represent its learning in useful artifacts. Together, the members take time as a community to “reflect on and self-design” their learning processes, so that they can continuously improve the CoP for each other.9

CoPs can vary greatly in form and size. They can form within an organization or connect individuals across various organizations. They may be formally recognized and supported by a budget, or relatively informal. Some are quite small in terms of the number of members or geographic boundaries, while others are large in number and global in reach.

Figure 2. Learning processes in a community of practicea

Figure 2 depicts four key learning processes involved in many CoPs.  In the middle of the figure is an oval representing the CoP or learning partnership, comprised of a domain, community, and practice.  On the left, an arrow pointing right depicts members bringing their practice into the CoP by sharing their stories and experiences.  On the right, an arrow pointing right depicts members pushing their practice forward and beyond the CoP, by exploring new ideas.  At the bottom, an arrow pointing up depicts members creating self-representation in the CoP by deriving lessons from the information in the CoP.  At the top, an arrow pointing down depicts members reflecting and self-designing their learning processes, so that they can continually improve the CoP for each other.

aUsed with permission. Wenger E, Trayner B. Personal e-mail communication. 21 September 2012. Resource list available at: http://wenger-trayner.com/map-of-resources. Accessed August 29, 2013.

While a CoP is typically driven by a core group of people who are passionate and devote time to helping to lead the community, not all the members necessarily participate equally. Multiple levels of participation usually exist, reflecting differences in members’ perspectives, needs, and ambitions. As illustrated in Figure 3 below, in addition to the core group that energizes the community, other active participants are recognized as practitioners and help define the community, even if they are not always in agreement about where the group is headed.10 Occasional participants are those who only participate when the topic is of special interest, when they have something specific to contribute, or when they get involved with a special project. Peripheral participants are those who have a sustained connection to the community, but are less engaged due to their recent arrival or lower level of personal commitment to the practice. Transactional participants are individuals who occasionally interact with the community to provide or receive a service or artifact (e.g., publication, Web site, tool), but are not members themselves. Members may move in and out these categories over the life of the community, even as more specialized subgroups or constituencies also may form within the larger community.

Wenger has identified the three key factors for successful CoPs: identification, leadership, and time. Accurately negotiating the domain (i.e., scope) of a CoP allows members to identify with its purpose and also to derive a new identity as a participant in the CoP. Ensuring that a core group of leaders steps up to nurture the community and “take care of logistics” guards against the loss of momentum and member interest. Finally, members are keenly aware of the other priorities that compete for their time, so it is important to ensure a “high value for time” ratio for members.11

Figure 3. Levels of participation in a community of practiceb

Figure 3 depicts the community of practice as a series of concentric circles, to represent the varying levels of participation that individuals can have with the CoP.  At the very center is a core group, which typically consists of coordinators and leaders.  Slightly less removed than that group is an active group of members, which often includes experts in the field.  Next comes occasional participants, including members who may check in with the CoP every once in a while.  Next are peripheral participants, which often includes beginners (either in the field as a whole or those who are new to the CoP) and lurkers - those who tend to observe the CoP without participating actively.  Finally, there are transactional individuals who may not participate in the CoP per se, but play a particular role in its operation.  These include support staff, sponsors, and clients.

bReprinted with permission. Wenger E, Trayner B. “Slide: Levels of participation.” http://wenger-trayner.com/resources/slide-forms-of-participation. Accessed August 29, 2013.

Examples of Web-Based CoPs

The concept of a Web-based or online CoP is not a new one; in fact, Web-based CoPs have been used to support audiences and purposes as varied as nursing,12 nurse midwifery,13,14 community health nursing,15 pre-service education,16 occupational therapy,17 equine science and management,18 orthopedic surgery,19 and oral medicine.20 Web-based CoPs have also been proposed as a tool for use in health policy implementation in low-income countries,21 pediatric chronic disease management,22 emergency clinical care,23 general practice,24 and mental health care provision in rural areas.25 Several representative examples of existing Web-based CoPs in health care and education are profiled below.

phConnect

phConnect (www.phconnect.org) is a professional networking and collaboration site for public health, sponsored by the CDC and its public health partners, with over 5,000 members as of August 2012. Although membership is open to anyone with experience or interest in public health, regardless of physical location, registration and creation of a member profile is required. Within phConnect, members can participate in one or more communities or launch a community focusing on a particular public health topic. Technical features of the phConnect Web site include discussion forums, an events calendar highlighting upcoming Webinar and in-person meetings that may be of interest to members, an announcements section, and a video library.

Research to Reality

Research to Reality (R2R, https://researchtoreality.cancer.gov) is an “online community of practice designed to bring together cancer control practitioners and researchers to discuss moving evidence-based programs into practice.”26 The program was developed and is supported by the NCI at the NIH. Technical features of the R2R Web site include discussion threads, cyber-seminars (i.e., Webinar), and an events calendar. Aside from offering members the chance to interact with each other directly, R2R also regularly generates content to foster conversation and discussion among members. For example, the “Featured R2R Partners” section of the Web site highlights members’ personal stories of moving research into practice, and the “Mentorship Program” section provides updates on six mentor-mentee pairs that are working together to implement an evidence-based intervention in their communities. See Appendix D for screen shots of three representative pages on the R2R Web site: Home, Discussions, and Cyber-Seminars.

TA&D Network

The Technical Assistance and Dissemination Network (TA&D Network) consists of approximately 45 centers and projects funded by the Office of Special Education Programs (OSEP) to help support implementation of the Individual with Disabilities Act (IDEA). The network currently hosts two Web sites for their CoPs. The first site, www.tacommunities.org, was custom-built and launched in 2007; its primary audience is the funded centers within the TA&D Network. The Web site features topic-based subgroups, searchable member profiles, discussion boards, and a document sharing area. See Appendix E for screen shots of the Web site. After several years, the Network decided to expand their audience to include a broader group of stakeholders, including families of children with disabilities. The second site, www.tadnet.ning.com, was built on the Ning platform and offers the additional features of a blog, events calendar, and video library. Both Web sites are currently functional and serving the TA&D Network CoP.27

IDEA Partnership

The IDEA Partnership is a program within the TA&D Network, jointly sponsored by OSEP, under the Department of Education, and the National Association of State Directors of Special Education, Inc. The Partnership works with Federal agencies, national organizations, State agencies, and stakeholder groups to build the capacity of States, districts, and schools for improving results for students with disabilities and to share best practices for implementation of IDEA.28 Their CoP Web site, www.sharedwork.org, provides the technical infrastructure to host many different individual communities within this interest area. Eight national-level communities, focused on broad topic areas such as school behavioral health and autism, are hosted and maintained by IDEA Partnership staff. Within those eight communities are 174 subcommunities organized by region, State, practice type, or organization. These subcommunities are initiated, organized, and maintained by members, and administrative duties such as adding and moderating content can be delegated from the Partnership staff to leaders of the subcommunities.29

Advancing Best Practices for Web-Based CoPs

The examples described here of Web-based CoPs in health care and other domains illustrate the feasibility of launching and operating such a tool. Beyond these practical examples, there are also resources to provide targeted support to those who are managing CoPs in specific domains or with the support of specific sponsors. These initiatives have effectively created a community of practice around communities of practice.

For example, the CDC is involved in advancing the science and practice of CoPs for public health. The Web-based CDC Communities of Practice Resource Kit offers resources for public health groups interested in launching a CoP, including a step-by-step guide and tools for planning, launching, sustaining, evolving, and evaluating a CoP.30 CDC also delivers this content in courses for CDC staff and at conferences and grantee meetings upon request.

Another resource is the Federal Virtual Community of Practice Group, which is a group of approximately 20 managers and coordinators of Federally-sponsored CoPs interested in exchanging ideas and information about best practices for managing these communities. NCI has been the primary organizer of the group, in collaboration with CDC. An in-person kickoff meeting was held in June 2012, and participants pledged to meet again to generate a community charter and discuss possible collaboration methods for the group.31

Existing Patient Registry Resources

Many of the essential elements of a CoP already exist around the domain of patient registries. The community of those who sponsor, manage, evaluate, and use data from patient registries includes individuals from such diverse backgrounds as government, private industry, payers, academic research, and clinical, patient, and patient advocate organizations. These stakeholders have a history of coming together to discuss methods, best practices, and share knowledge around patient registries. This knowledge exchange has traditionally taken place in relatively fragmented and infrequent formats, including reading and writing published peer-reviewed articles and other guidance publications (e.g., “Registries for Evaluating Patient Outcomes: A User’s Guide”), networking at professional conferences, Webinar and working groups (e.g., those hosted by AHRQ and the Drug Information Association [DIA]), and informal conversations with colleagues.

More recently, some attempts have been made to provide a more structured opportunity for knowledge-sharing about registries. For example, one initiative at the forefront of advancing knowledge-sharing about patient registries is the National Quality Registry Network (NQRN). Staffed by the American Medical Association through the Physician Consortium for Performance Improvement, NQRN is a “voluntary network of private and public registry stewards and other stakeholders interested in advancing the development and use of registries to evaluate and improve patient outcomes.”32 One of its stated goals is to “advocate for and support a learning network to accelerate national progress on registry development, growth, and use.”33

While this network may confer some of the same benefits to participants as a CoP centered on patient registries, the NQRN does not yet have an online presence to make its resources broadly available.

Findings

Need for a Web-Based CoP on Patient Registries

In general, there is a high level of interest in this topic and stakeholders were eager to discuss it. The response to the stakeholder meeting invitation was very strong; all in-person attendee spots were filled, some attendees participated by phone, and a waiting list was formed.

At the meeting, stakeholders were asked to describe the current knowledge-sharing environment in which they conduct their work and whether or not there is a need for a Web-based CoP on patient registries. As mentioned above in the Background Research section, stakeholders reported several ways in which they currently learn about patient registry practices and interact with others regarding this subject. These included:

  • Networking, brainstorming, and conversing with colleagues in person and via email
  • Attending professional/specialty conferences
  • Attending project-specific meetings for registries (e.g., investigator meetings)
  • Reading peer-reviewed journal articles on the topic
  • Reading other methods-focused publications (e.g., “Registries for Evaluating Patient Outcomes: A User’s Guide”)
  • Participating in working groups or learning networks in person and via teleconference (e.g., DIA, Physician EHR Coalition, NQRN)
  • Trial and error; learning from one’s experience and mistakes

After describing their current knowledge-generating and knowledge-sharing practices, stakeholders clearly articulated the consensus that a Web-based CoP was needed and would be welcomed in the domain of patient registries. In fact, no dissenting opinions were expressed; much of the discussion from stakeholders centered not on whether there should be a Web-based CoP for this community, but the details around what that Web-based CoP should look like and how it should be implemented.

Value Proposition

The second portion of the in-person meeting focused on understanding the value proposition for the creation of a Web-based CoP. Again, there was a clear consensus: stakeholders expressed that the added value of such a Web-based CoP would primarily be determined by the extent to which it was relevant to their work and helped them to do their work better, faster, or more easily. In addition to this overarching goal, stakeholders stated that they would find value in a Web-based CoP that meets the following goals:

  • Facilitates networking and interaction between various stakeholders and the building of relationships and trust. Provides a place where stakeholders can come together.
  • Helps members identify practical ways to improve their work practices, with the ultimate goal of improving patient care and outcomes and effecting change on medical practice.
  • Serves as an organized system for learning and information sharing.
  • Supports collaboration within projects and across organizations. Facilitates collaborative problem-solving.
  • Overcomes “silos.” Facilitates efficiencies in resource use and reduces duplication of efforts.
  • Serves as a collective voice for registries (e.g., on national policy, to impact the electronic health record [EHR] industry)

If a Web-based CoP is to meet the goal of helping its members do their work better, its content (e.g., discussions, Webinar, documents and publications) must be timely and relevant. Stakeholders noted that managing this content would require some level of staffing by individuals with knowledge about and experience with patient registries.

The question of which topics will be covered in a Web-based CoP can be decided by the sponsor or members. Stakeholders expressed interest in seeing a variety of topics addressed, as shown below in Table 1. The topics are organized by the level of interest they generated among stakeholders at the in-person meeting, as measured by the frequency with which they were mentioned. Those topics classified as eliciting “more interest” from stakeholders were mentioned—either verbally or in written notes—by five or more different stakeholders during the meeting. Topics of “some interest” were mentioned by two to four stakeholders, and topics of “less interest” were mentioned by a single stakeholder.

Table 1. Topics of interest for stakeholders in a Web-based CoP on patient registries
  More interest Some Interest Less Interest
Topics of Interest
  • Methods and best practices
  • Standards (e.g., data elements, outcome measures)
  • Funding
  • Privacy and security; data integrity
  • Technologies for registries
  • Interoperability of registries with other systems (e.g., EHRs), and the role of vendors
  • Meaningful Use reporting (including health information exchanges and regional health information organizations)
  • Lessons learned
  • Legal and regulatory issues
  • Informed consent (e.g., for those with disabilities)
  • Data ownership, sharing, access, and use
  • Developing a registry (e.g., protocol, structure, and design)
  • Developing and advancing research agendas (e.g., Coverage with Evidence Development)
  • Registry sustainability and preventing registry fatigue
  • Adverse event reporting
  • Data quality, curation, and assurance
  • New and emerging practices
  • When is a registry appropriate? When should a registry be started?
  • When should a registry end?
  • Institutional Review Board reporting
  • Ethics in designing, conducting, and using data from patient registries
  • Patient-reported outcomes in registries
  • Differences in the registry environment across geographic regions
  • Branding and naming a registry
  • Risks and benefits of operating internationally
  • Defining a registry’s focus
  • Clinician concerns about participation
  • What knowledge is available from various data sources (e.g., claims data)

Key Design Issues

The last portion of the in-person stakeholder meeting was devoted to discussing with stakeholders what specific features they wanted to see in a Web-based CoP on patient registries, and what key design issues they perceived to be important. As the discussion continued, the features and issues mentioned by stakeholders began to fall into five broad categories: technical features, governance, target audience, levels of participation, and sustainability. These issues are summarized below, and are discussed in more detail in the Recommendations section of this report.

Technical Features

There was consensus among stakeholders that the Web-based CoP should be hosted on an independent Web site. To maximize sustainability and efficiency, stakeholders agreed that the Web site should use an off-the-shelf technology platform or Web hosting service, rather than a completely custom-built solution; sufficient resources should be devoted to maintaining the Web site, paying any related fees, and securing storage space.

Stakeholders preferred that the structure of the Web site allow for the creation of sub-forums or sub-communities. These could be organized by discussion topic, clinical area, specialty, member role, or other category and should enable discussion on both broad and specific topics. Stakeholders stated that members should be able to participate in threaded conversations and question-and-answer sessions and that dedicated staff should be available to monitor and moderate discussion boards, guide discussion and solicit comments when needed, and enforce appropriate use of the Web site. Stakeholders also agreed that anonymous participation in the Web-based CoP should not be allowed; members should provide basic contact information, and Web site administrators should have access to a member directory.

Stakeholders agreed that the Web-based CoP should serve as a central source of information about patient registries and that resources should be made available to members that may include frequently asked questions (FAQs) and a searchable index of relevant references (i.e., citations of peer-reviewed publications, journals, Web sites, and published guidance documents). Stakeholders suggested that these resources be updated by staff on a regular basis (e.g., quarterly) with new, relevant registry-related resources and links.

In addition to its role in referring members to external information sources, stakeholders saw the Web-based CoP as facilitating the generation and dissemination of information that advances the science and practice of patient registries. For example, periodic Webinar could be organized by staff, with topic areas ranging in breadth from introductory (e.g., “Registries 101”) to more advanced (i.e., for members with more registry experience); previous Webinar should be archived in a video library.

Governance

Stakeholders stated that the Web-based CoP should be governed by an advisory group or steering committee, and that the sponsor should be recognized as a trusted entity, perceived as unbiased and able to serve as an honest broker. Stakeholders noted that they could easily see AHRQ filling this role. Stakeholders also mentioned a public-private partnership as a possible sponsorship model and recommended that a disclosure of the sponsor’s and governing body’s commercial conflicts of interest be posted publicly.

Stakeholders suggested that a charter should be written to guide governance of the Web-based CoP. The charter should define the purpose and scope of the CoP, provide the definition of a patient registry, differentiate this initiative from other existing initiatives in the field (e.g., the RoPR), and describe etiquette and appropriate use of the interactive portions of the Web-based CoP (e.g., policies on spam, self-promotion, and disclosing personal health information).

Target Audience

Stakeholders noted that the intended audience of the Web-based CoP should represent the wide variety of stakeholders that are currently involved in patient registries, including registry participants (i.e., patients); registry designers, managers, and operators; providers and clinicians; researchers (including those who use registry data); industry and registry sponsors; regulatory bodies; and payers. Stakeholders suggested that use cases of typical users of the CoP should be identified and should inform plans for marketing the Web-based CoP.

Stakeholders also raised questions related to the scope of the Web-based CoP’s target audience and content. Would its target audience be primarily U.S. stakeholders, or would it extend to international stakeholders? Should access or membership be restricted based on the geographic area in which a member conducts their work? Similarly, would the Web-based CoP be targeted only to those who have experience with patient registries, or would the target audience include those with interest in registries but little or no registry experience (e.g., patients, members of the public)? Should access be limited to a specific group of stakeholders based on topical interest? While stakeholders did not arrive at a clear consensus on these questions, they agreed that the sponsor of the Web-based CoP would need to address these questions.

Levels of Participation

Stakeholders recognized that while a CoP relies on the generation and sharing of knowledge among its members, there may be some situations in which members would be reluctant to share information. For example, members who are employed in the pharmaceutical or medical device industry may feel the need to censor or limit their participation in knowledge-sharing activities to avoid disclosing proprietary information. More experienced members who could offer valuable advice on “what not to do” based on their past experience may be reluctant to do so if they perceive that sharing this information would be detrimental to their (or their organization’s) reputation, or equivalent to admitting a mistake. In other words, “If knowledge is power, why would anyone want to share it?”34 Stakeholders identified this as a potential barrier to successful implementation of a Web-based CoP, and advised that the sponsor proactively consider how to address this issue.

Sustainability

Finally, stakeholders noted that members who devote time and effort to participating in a Web-based CoP will want reasonable assurance that the CoP will be available to them in the long term. Stakeholders advised that the question of a sustainable funding source be addressed upfront by the CoP’s sponsor.

Recommendations

Based on the background research and stakeholder discussions, it is clear that there is a very strong interest in a Web-based CoP on patient registries and that a Web-based CoP is a practical, feasible, and timely idea. Stakeholders have a broad range of questions about creating and using registries that they want to discuss in this type of forum. The software required to conduct such a Web-based CoP is readily available “off-the-shelf” so that the technical focus would be on selecting which features to use and apply rather than building custom software. The U.S. Department of Health and Human Services and other Federal agencies already sponsor numerous Web-based CoPs, providing precedents regarding regulatory compliance and general approach. Furthermore, stakeholders interested in patient registries see AHRQ as a logical choice for advancing this effort, given its reputation as an unbiased, knowledgeable, and trusted leader in the field of observational research methods, quality improvement, and patient registries.

In keeping with the vision laid out by stakeholders, the creation of a Web-based CoP on patient registries that would provide a mechanism for persons interested in starting registries or using existing ones to engage in discussions with their peers and other subject matter experts about a broad range of registry-related topics is recommended. The forum would use existing technologies, which would require the configuration of existing software but minimal, if any, new software development. Participants would self-identify and register as members to participate in password-protected CoP activities. The CoP would provide access to a library of documents and other resources appropriate to the topic of patient registries and would leverage existing resources on registry best practices. Members could initiate discussion topics and invite others to join subgroups, as needed, to create meaningful and productive interactions. Web-based interaction would be supplemented by conference calls, Webinars, and in-person meetings to the extent that the sponsor’s budget may allow. Such related activities could also be supported by other CoP members and institutions or related initiatives focused on registry best practices. An advisory group would provide input to the CoP sponsor(s) regarding changes that may be needed over time. Dedicated, professional staff would be responsible for monitoring all CoP activities, organizing materials submitted, and facilitating interactions, as needed. These and other features of the forum are described in more detail below, and the recommendations are summarized in Tables 2, 3, 4, and 5.

Goals and Objectives

Stakeholders are looking primarily for a tool that will be broadly useful, relevant to their work, and help them to do their work better, faster, or more easily. Secondary goals that stakeholders have for the CoP include facilitating networking, interaction, and collaboration; improving efficiencies in resource use and reducing duplication of effort; helping members identify practical ways to improve their work practices, with the ultimate goal of improving patient care and outcomes; providing an organized system for learning and information sharing; and serving as a collective voice for registries on the national stage.

To meet these goals, the Web-based CoP should provide an organized system for members to share information and learn about patient registries, guided by the input of an advisory group or professional staff. Core components of this system should include a resource library that provides members with easy access to documents and other relevant materials about patient registries; a series of Webinars and related discussions examining good registry practices and research methods, building off the content contained in “Registries for Evaluating Patient Outcomes: A User’s Guide” and other related materials; and a support system to encourage the formation of working groups to explore new and emerging issues related to the development and use of patient registries for research, quality improvement, and safety surveillance.

These goals and objectives are in line with AHRQ’s priorities as a prominent stakeholder in the patient registry community. AHRQ’s mission is “to improve the quality, safety, efficiency, and effectiveness of health care for all Americans.”35 AHRQ has invested significant resources in developing good practices for registries and improving the usefulness and quality of registry data. AHRQ is also developing the RoPR system to promote collaboration, reduce redundancy, and improve transparency in registry research. Taken together, these registry-related activities have established AHRQ as a major supporter of the use of registries to conduct practical, high-quality clinical research. The development of a Web-based CoP on patient registries can be viewed as a continuation of AHRQ’s support for registries and as promoting quality and efficiency in health care research. Based on this rationale, recommendations for potential goals and specific objectives of a Web-based CoP on patient registries are presented in Table 2.

Table 2. Summary recommendations for goals and objectives
Goals Objectives
Establish a Web-based CoP using the principles and recommendations described in this report. The primary goals of the CoP should be consistent with those voiced by stakeholders:
  • To improve stakeholders’ ability to share knowledge and experiences.
  • To foster Web-based networking, interaction, and collaboration among stakeholders.
  • To enable stakeholders to work better, faster, or more efficiently in designing, operating, analyzing, and evaluating patient registries.
The Web-based CoP should pursue these goals by way of the following objectives:
  • Provide an organized system for sharing information and learning about patient registries guided by the input of an advisory group and professional staff.
  • Provide a resource library to offer easy access to documents and other relevant materials about patient registries.
  • Develop and facilitate a series of Webinars and related discussions examining good registry practices and research methods, building off of the content contained in Registries for Evaluating Patient Outcomes: A User’s Guide and related materials.
  • Encourage and support the formation of working groups and ad hoc projects involving CoP members to explore cutting-edge issues related to the development and use of patient registries for research, quality improvement, and/or safety surveillance.

Technical Features

The Web-based CoP should use innovative technology to enable participation and contributions from members. The technical features described below are known as Web 2.0 concepts; that is, “Web applications that facilitate interactive information sharing, interoperability, user-centered design, and collaboration on the World Wide Web.”36 In addition to the ability to post and share documents, Web 2.0 concepts for information sharing may include the use of wikis, blogs, and mashups, each providing a unique opportunity for collaborative interaction over the Web. Use of these collaborative tools will serve the stated goal of stakeholders to enable them to do their work better, easier, and faster. For definitions of many of the terms used below, see Appendix A, “Glossary of Terms.” Table 3 provides a summary.

Web Site

At the very minimum, a Web-based CoP on patient registries will require an independent Web presence through a Web site. Resources should be devoted to physically maintaining the Web site, including paying relevant fees and securing storage space. Some CoPs function without an independent Web presence and may share knowledge and collaborate through email communication and live and remote meetings. Others use the infrastructure of existing CoP Web sites, which allow members to create subgroups or subcommunities on a particular topic. However, these approaches are limited in the level of control one has over the physical infrastructure of the meeting facilitation. An independent Web site can provide information to those not familiar with the CoP and help them conceptualize the purpose of the CoP, what it can offer them, and how they can contribute. A Web site can also bring inbound marketing through search engine results, expanding the potential audience of a CoP. Finally, a Web site can offer concrete tools, which are discussed further in the subsequent sections, to facilitate interaction and collaboration among members.

Based on the background research and stakeholder perspectives, the Web site should be hosted and operated using an existing, off-the-shelf technology platform, rather than a completely custom-built solution. Platforms such as these use templates and modular structures to “plug and play” functionality for their clients, with minimal new coding required. This introduces efficiencies in the time, human resources, and funding required to launch and maintain a Web site. Many existing Web-based CoPs use off-the-shelf Web hosting platforms to launch and maintain their Web sites—tasks that would otherwise require a staff of in-house Web developers. TA&D Network uses the Ning platform (www.ning.com),37 the IDEA Partnership uses Liferay (www.liferay.com),38 and Research to Reality uses Drupal (www.drupal.org).39 The technology platform chosen by the sponsor should be able to host the features recommended below and return basic metrics to allow the sponsor to monitor activity associated with use of the Web-based CoP.

If additional resources are available, AHRQ could implement customizations to the template solutions offered by these vendors. The Web site could also be integrated or linked with other registry-related Web resources, including the RoPR.

Discussion Boards and Chat Forums

Discussion boards and chat forums are technical features of a Web site that facilitate threaded conversations and question-and-answer sessions among members. The primary purpose of these features is to facilitate and encourage interaction among members. At minimum, the technical infrastructure for these features should be present in the Web-based CoP. Discussions and chats can be driven completely by members and do not necessarily need to be moderated by CoP staff. However, resources permitting, the CoP staff should be involved in monitoring and moderating discussion boards to guide discussion, solicit comments, and enforce the appropriate use of the CoP. Experts involved with existing CoPs emphasized that some level of staff involvement is important to encourage participation and keep members engaged. The boards and/or forums should be organized into subforums by discussion topic, clinical area, specialty, member role or other category, allowing for discussion on broad and specific topics. If sufficient resources are available, content within the forum should be cataloged such that it is searchable by certain characteristics (e.g., keyword, phase in the registry lifecycle, intended audience).

Resource Section

A section of the Web-based CoP devoted to resources would serve as a central location for information on the topic of patient registries. This section should serve as both a library and a signpost, allowing both for the posting of complete files and documents and the posting of hyperlinks that direct the user to relevant external Web sites or files. The section may also include frequently asked questions (FAQs) and citations of peer-reviewed publications, journals, Web sites, and published guidance documents.

In order to remain relevant and up-to-date, the CoP staff should regularly (e.g., quarterly) review and update the content of this section to incorporate new registry-related resources and links. A process should exist to accommodate the addition of resources that are suggested by members. Content in the library should be indexed and searchable to make it as easy as possible for members to use the section.

If more resources are available, the resources section could be reviewed and updated on a more frequent basis. The scope of the content in the section could be expanded to include a library of data elements, definitions of commonly-used terms in registry science, templates for registry protocols and informed consent forms, and training and education modules. A document sharing and management area could be made available that allows members to upload documents for other members to view.

Webinars and Teleconferences

In addition to its role in referring members to external information sources, the Web-based CoP should facilitate the generation and dissemination of information that advances the science and practice of patient registries. Webinars (with both an audio and visual component) and teleconferences (with an audio component only) are common methods for hosting live meetings with remote attendees. The CoP staff should organize periodic Webinars and/or teleconferences, with topic areas ranging from introductory (e.g., an overview of the role of registries in evidence development) to intermediate (e.g., registry design principles) to advanced (e.g., standards and best practices for evaluating registries). Speakers may include CoP staff or invited guest speakers, such as authors of publications related to registries. Previous Webinars (and/or the slide presentations used at them) should be archived in a video library and available for members to view at a later date. A calendar of events can be posted to make members aware of upcoming Webinars and teleconferences.

Member Directory

A member directory should be available to CoP staff to facilitate administration of the Web site and its members. The directory should contain the information collected from members when they registered to participate in the CoP. Because of potential concerns about member privacy, information on participation history (e.g., an audit trail) should not be included in the directory. The CoP should consider whether to make the directory available to other members; this feature may be useful to members, but it also raises privacy concerns. One option would be to offer members the choice at enrollment to disclose their information. The implications of making the directory accessible to members may also vary by sponsor (e.g., a Federal sponsor vs. a private entity sponsor) and should be investigated.

Additional Features

The following features were not mentioned by stakeholders as essential features for a Web-based CoP. However, existing CoPs have used these features to enhance utility and maintain member engagement. While they are less critical than the five features mentioned previously in this section, they could be implemented if and when resources are available.

Outgoing Communications to Members

Outgoing communications can help to maintain engagement and interest among members of the CoP. These communications can inform members about new content added to the CoP, upcoming events, or changes to the Web site itself. Communications can be explicitly requested by members (e.g., by registering to receive notifications when updates are made to forum content) or implicitly allowed (e.g., if members give their permission when registering for the CoP to receive periodic newsletters). Regardless, members should be able to opt out of communications at any time. Ideally, members should also be able to manage the frequency with which they receive communications from the Web-based CoP. The communications can also be implemented in a passive (e.g., podcasts recorded and posted on the CoP Web site) or active way (e.g., emails sent directly to members). Active communications to members can vary in format, including email, RSS, or Twitter.

Wiki

Wikis provide a community-based mechanism for creating and editing Web pages that are interlinked and developed over time to share information and to access that information quickly and easily through hyperlinks. They are essentially a community-built knowledge base around a particular topic. When appropriately maintained and moderated, wikis can be powerful resources for learning and knowledge sharing. A wiki supported by the Web-based CoP should allow members to contribute content and should be moderated by CoP staff.

Blog

Blogs provide a forum for regular and frequent communication on a topic and an opportunity for replies and commentary by participants on the subject under discussion. CoP staff (or invited guests) can create individual blog posts on a particular topic; the writing tone for blog posts is usually similar to that of an editorial. The blog posts should be published on the CoP Web site, usually in a serial way, and members should be able to comment openly on individual posts. This format is generally advantageous when attempting to generate discussion and/or assert opinions.

News

Many online CoPs have a section of their Web site devoted to news. This is typically one or more pages where members can view upcoming events and recent developments in the field of interest. News items can be organized by sponsor, topic, or date, and can include events or developments sponsored by the Web-based CoP as well as those external to the CoP. A well-organized and updated news section can serve to draw traffic to a Web site and maintain interest among members.

Mashup

Web mashups are applications that combine data or functionality from multiple sources and display them in a new way. Mashups may take the form of informative data dashboards (which aggregate and display information from various sources in charts, graphs and tables) or more interactive mapping mashups (which pull data from different sources to display geographically on a map). When used creatively, mashups can add value by showing relationships between data that have not previously been aggregated and compared.

Table 3. Summary of recommendations for technical features
Recommendations Technical Features
Minimum Recommendations
  • Establish a Web site, and devote resources to physically maintaining it. Host and operate the Web site using an existing, off-the-shelf technology platform, rather than a completely custom-built solution.
  • Host discussion boards and chat forums, and designate staff to monitor and moderate them. Organize the discussion boards into subforums by discussion topic, clinical area, specialty, member role or other category, allowing for discussion on broad and specific topics.
  • Host a resources section that allows staff to post complete documents and hyperlinks to external Web sites or files; regularly review and update this content. Content in this section should be indexed and searchable.
  • Organize quarterly Webinars and/or teleconferences, covering a variety of topics. Archive previous Webinars in a video library for members to view at a later date.
  • Maintain a member directory that is available to CoP staff and contains minimal information collected from members. Do not include information on participation history in the directory.
Recommendations for Additional Features, if Resources Are Available
  • Implement customizations to the Web site. This could include integration with other registry-related Web resources, including the Registry of Patient Registries (RoPR).
  • Catalog content within the discussion boards such that it is searchable by certain characteristics (e.g., keyword, phase in the registry lifecycle, intended audience).
  • Review and update the resources section on a more frequent basis. Expand the scope of content and establish a document sharing area that allows members to upload documents for other members to view.
  • Organize monthly Webinars and teleconferences covering a wider variety of topics.
  • Investigate the implications of and potential approaches for making the member directory accessible to members in addition to CoP staff.
  • Manage outgoing communications to members, with information about new content added to the CoP, upcoming events, or changes to the Web site itself. Members should be able to opt out of communications.
  • Maintain and moderate a wiki.
  • Host a blog. Allocate time for CoP staff to write blog posts and manage posts by invited authors.
  • Host and maintain a news section with upcoming events and recent developments in registry science.
  • Consider incorporating a mashup to aggregate and display relevant data from different sources.

Governance

Governance of a Web-based CoP includes both the sponsor or primary source of financial support and the governing body that oversees the day-to-day operations of the CoP. The choice of sponsor, particularly whether or not the sponsor is an agency of the Federal government, has implications for the regulatory requirements for the CoP. A governing document will need to guide the governing body in defining the purpose and scope of the CoP. The stakeholder discussions and background research conducted for this project focused on questions related to the CoP sponsor and the purpose and composition of the governing body. Table 4 provides a summary of governance recommendations.

Sponsor or Funding Source

Based on stakeholders input, the Web-based CoP should be sponsored by one or more entities that are perceived as unbiased and able to serve as an honest broker for the content contained in the CoP. Possible sponsorship models include a single sponsor or a public-private partnership, in which a public agency such as AHRQ collaborates with a private entity (e.g., a nonprofit organization or commercial business) to fund and support the CoP. To promote transparency, a disclosure of the sponsors’ commercial or financial conflicts of interest should be posted on the Web site and available to the public. If there are no commercial conflicts of interest, this should be stated explicitly.

Existing CoPs can serve as examples of how to structure the sponsorship of a Web-based CoP. For example, Research to Reality is supported solely by the NCI at NIH.

Considerations for a Federal Sponsor

Web sites funded or sponsored by Federal agencies are subject to regulations that do not necessarily apply to Web sites sponsored by private companies or other organizations. If the funding structure or governance of the Web-based CoP discussed in this report includes AHRQ, AHRQ’s designee, or another Federal agency, that will have implications for the regulations to which the Web-based CoP is subject. Appendix F summarizes specific regulations that are relevant to other existing Web-based CoPs sponsored by Federal agencies. Appendix G contains a recent memo from the Office of Management and Budget which clarifies how the Paperwork Reduction Act of 1995 should be applied to many of the technologies that may be used in a Web-based CoP (e.g., social media and Web-based interactive technologies).

Financial Sustainability

One concern raised by stakeholders was the availability of sustainable funding for the Web-based CoP. The stakeholders especially noted that members devoting time and effort to participating in a Web-based CoP would probably want reasonable assurance that it would be available to them in the future. If members know or suspect that the underlying funding is short-term or uncertain, they may be hesitant to invest their time and energy into the CoP.

Ideally, before a Web-based CoP is launched, the sponsor should have long-term plans for maintaining funding in the future. If value determinations need to be made to prioritize future funding, the technological infrastructure of the CoP should be maintained as a first priority. Secondary to that, adequate staff should be maintained to answer questions raised by members, guide members to resources, and develop content for the CoP. Finally, if further funding is available, it should be used to maintain any live Webinars or teleconferences that are hosted by the CoP. Such long-term plans will bring stability to the CoP and help to engender confidence in members that they are engaging in a worthwhile enterprise.

Governing Body

Stakeholders clearly articulated a preference for transparent, unbiased leadership for the Web-based CoP. Regardless of its sponsor or funding source, the CoP should be governed by an independent body such as an advisory group or steering committee, comprised of representatives from multiple relevant registry stakeholder groups. The purpose of this group would be to guide the overall direction and activities of the CoP. The governing body should meet regularly (e.g., quarterly) to assess the state of the Web-based CoP and address any issues that have arisen in its day-to-day management.

Governing Charter

The governing procedures of the Web-based CoP should be transparent and consistent with stakeholder priorities. A charter should be written to guide the governing body. The charter should define the CoP’s purpose and scope (which may include providing the definition of a patient registry), differentiate the CoP from other existing initiatives in the field (e.g., the RoPR), and describe policies and procedures, including etiquette and appropriate use of the forum (e.g., spam policies, posting of protected health information). The charter should be reviewed on an annual basis and revised when necessary.

Communication Plan

Communication activities for the Web-based CoP should be carefully planned and supported with resources. The governing body should articulate a communication plan that outlines the situations in which active and passive communications to members will be generated, along with the frequency and general content. This plan can either be part of the charter or a separate document, which can be updated more frequently than the charter, as needed.

Types of communications specified in this plan may include those explicitly requested by members (e.g., by registering to receive notifications when updates are made to forum content) or implicitly allowed by members (e.g., if members give their permission when registering for the CoP to receive periodic newsletters). They may also include passive (e.g., podcasts recorded and posted on the CoP Web site) or active (e.g., email, RSS, or Twitter) communications.

Table 4. Summary of recommendations for governance
  Recommendations
Characteristics
  • The Web-based CoP should be sponsored by one or more entities that are perceived as unbiased and able to serve as an honest broker for the content contained in the CoP. A disclosure of the sponsors’ commercial or financial conflicts of interest should be posted publicly. If the sponsor includes a Federal agency, consider the regulations to which the Web-based CoP will be subject, as described in Appendix F.
  • Create a long-term plan for maintaining funding.
  • The CoP should be governed by an advisory group or steering committee comprised of representatives from multiple relevant registry stakeholder groups. This group should meet regularly to assess the state of the CoP, address any issues, and guide the overall direction and activities of the CoP.
  • Write a charter to guide the governing body. The charter should define the CoP’s purpose and scope, differentiate the CoP from other existing initiatives in the field, and describe the policies and procedures. The charter should be reviewed on an annual basis and revised when necessary.
  • Articulate a communication plan that outlines the situations in which communications will be sent to members, along with the frequency and general content
 

Management

Ongoing management of a Web-based CoP should be the responsibility of the governing body or its designees. Management tasks should be guided by the policies and procedures set forth in the charter, which defines the scope of the CoP (in membership and topical content) and plans for marketing or promoting the use of the CoP. Two important questions are: (1) who will be technically permitted to view and participate in the Web-based CoP? and (2) who will be the CoP’s target audience? These questions are addressed below. Table 5 summarizes recommendations for management.

Membership and Access

The CoP must define who will be permitted to view and participate in the community. In the design of anyWeb site, there are typically two levels of user access: public and restricted (or “member”). At one end of the spectrum, access can be very open and fluid, and any member of the public has access to all areas of the Web site. This model is appropriate for sites that have a primary purpose of sharing information with the public (e.g., marketing a private company, disseminating news). At the other end of the spectrum, access can be very limited or nonexistent for the general public, and most or all of the Web site content can be visible only to members. A member is usually defined as an individual who has registered with the Web site previously (often providing basic information such as name and email address) and has created a username and password which allows them to log in and access areas of the Web site that are not accessible to nonmembers. In this scenario, the general public may only see a log-in screen when they visit the Web site. This model is appropriate for sites where the primary purpose is to disseminate or collect information from a limited, controlled group of people (e.g., Web-based data entry for a patient registry). Registrations for membership can be regulated to allow only members with certain characteristics (e.g., a study ID), limit the total number of members, or filter out “spam” or “bot” member registrations.

Many Web sites and Web-based CoPs manage membership and access in a manner that falls somewhere between these two extremes. Some areas of the Web site are available to the general public, and some areas or functions are only available to members. For example, the blog articles and subsequent comments on a Web-based CoP may be visible to everyone, but only members may submit comments to the blog.

The Web-based CoP on patient registries should follow this hybrid approach. To be consistent with the goal of sharing information about patient registries, CoP Web pages and resources should be available for the general public to view. A registration system that collects basic contact information about registrants should be implemented, and registered members should have a username and password to log in to the Web site (refer to the “Sponsor or funding source” section of this paper and Appendix F to review the implications of data collection in a Web site sponsored by a Federal agency). Members should have access to more advanced features within the CoP, which may include submitting comments to blog posts, participating (i.e., submitting messages) in a chat or discussion forum, and receiving outgoing communications from the CoP. Stakeholders stated a preference that anonymous participation should not be permissible in the Web-based CoP. By requiring users to register before contributing content, a measure of user accountability is introduced.

If more resources become available, a level of monitoring can be introduced to reduce “spam” or “bot” member registrations. In line with the CoP’s objectives to foster collaboration and interaction, member registration should not be limited in any other way (e.g., by total number of members or by any member characteristic such as education level, employer, geographic location, or specialty).

Target Audience

Although membership in the CoP should be open to anyone who registers, the target audience, or the user community that AHRQ most wants to assist and who would most benefit from such a Web-based CoP should still be defined. By defining a target audience, it is then possible to identify the particular information needs of various possible users, the advantages and disadvantages of focusing the audience narrowly or more broadly, and the design and programming implications of trying to meet these various needs. For example, registry sponsors might see the forum as a place to discuss their common concerns about funding and hosting registries; practitioners and health care providers who input data may want to discuss usability and benchmarking reports; researchers may want to identify data sources and observational study methods; and payers and consumers may want to focus on how to obtain and use comparative information about health care services to guide their purchasing decisions.

Based on the background research and feedback from stakeholders, the Web-based CoP should be flexible enough to engage and serve the interests of at least the following types of stakeholders: registry participants (patients); registry designers, managers, and operators; providers and clinicians; researchers (including those who use registry data); industry and registry sponsors; regulatory bodies; and payers. Stakeholders articulated that these groups are the ones that are most active and engaged already in the patient registry community and the ones that would benefit most from a Web-based CoP.

Depending on the resources available and the interest expressed among stakeholders, the Web-based CoP could expand its target audience to include public health professionals, patient advocacy groups, government, professional societies, practice-based research networks, lay people, and educators. While these groups were mentioned as possible users of the CoP, the consensus among stakeholders was that they would be secondary users to the groups mentioned above, and that the CoP should seek to meet their specific needs only after the needs of the primary groups have been met.

Topical Scope

Once the target audience has been determined, at least two questions of scope should be considered. First, what should be the scope of topics addressed by the content in the Web-based CoP? Content and resources that the CoP provides to its members (such as Webinars, teleconferences, discussion forums, blog posts, etc) must necessarily address a particular topic. Aside from the broad banner of “patient registries” under which the CoP will be organized, how should these specific topics be chosen? Should the generation of new topics be driven by the sponsor and administrators of the CoP, or by members?

Stakeholders recommended that the sponsor of the CoP allow members to be the primary drivers for the topical direction of the Web-based CoP. Table 1 presents a list of specific topics that stakeholders have stated they are interested in seeing addressed in the Web-based CoP. This list could inform the initial development of a CoP. The CoP should be organized in a way that allows members to initiate new topics in discussion forums and participate in the decision of which topics will be the focus of Webinars, teleconferences, and other resources.

In particular, it is relevant to mention the experience of TACommunities.org. The first version of this Web-based CoP, which was initiated in 2002, approached topic generation in a top-down manner. The sponsor created features centered on specific topics and then made them available to members for use. While the CoP enjoyed moderate member participation during this period, it was not until the CoP was reorganized in 2007 and topic generation became more member-driven that participation grew. Once members could create their own discussion forums on topics that were timely and relevant to them, participation increased substantially.

Geographical Scope

The second question of scope that should be addressed is to what extent the Web-based CoP will be focused on the United States or international interests. Registries are increasingly being used for research, evidence development, and quality improvement worldwide.40,41 This international interest in registries may need to be balanced with the fact that the Web-based CoP may be sponsored and financially supported by an agency of the U.S. government.

To balance these needs, some aspects of the Web-based CoP should be focused on the patient registry environment in the United States. For example, the primary language for the Web site and related materials should be English. In many other aspects, however, the CoP should be flexible enough to accommodate international interests. For example, membership should not be limited to individuals located in the United States. Similarly, content topics that are international in nature, especially if they are suggested by members themselves, should be accommodated in the CoP.

Marketing

Marketing the Web-based CoP could be approached in a number of ways. At minimum, the sponsor should devote limited resources to promoting awareness and use of the forum (e.g., press release at launch, notification via email distribution lists). However, since the strength of any CoP is the quality and quantity of member participation, the sponsor should develop a marketing plan for the Web-based CoP and allocate resources accordingly. The sponsor should identify use cases of typical users of the Web-based CoP and use this information to inform the marketing plan. If more resources are available, the CoP could be promoted via social media, proactive outreach to professional organizations and patient groups, and/or search engine optimization. The sponsor may also choose to contract with a professional marketing firm.

Resistance to Share Certain Information

As mentioned in the Key Design Issues section of this report, one concern mentioned by stakeholders was the possible resistance of some individuals to participate in the CoP or, once they are members, to share certain information. This resistance could threaten the effectiveness of a CoP, which relies on the generation and sharing of knowledge among members.

However, research on the subject has suggested that when individuals generally perceive knowledge as a public good to be exchanged and shared rather than a private good to hoard or barter with, they are more likely to participate in a CoP out of community interest, generalized reciprocity or altruism.42 The sponsor should encourage this viewpoint while marketing the CoP and address the issue proactively by presenting the Web-based CoP as a tool for mutual engagement and reputation building. Participation in a CoP is not a one-way street, and those contributing knowledge and experience also stand to benefit from the combined knowledge and experience of their fellow members. Participating in a CoP can also be leveraged as a way to make one’s self (and one’s work) known to a wider audience, building a reputation by interacting with members and get to know one another over time.43

Monitoring and Assessment

Monitoring and assessing the health and impact of a CoP allows the sponsor to understand how the CoP is evolving over time, determine the extent to which it is meeting its goals, and gain insights for refining and improving it, as needed. CoPs can be monitored in two ways: through the activity that they facilitate among members and through their effectiveness as perceived by members.44 The activity in a Web-based CoP can often be monitored through automated reports that track and display new member registrations, overall site visits, and interaction among members (e.g., discussion forum posts). Measuring the effectiveness of a CoP typically requires asking users to provide feedback about relevance, usability, and value through questionnaires, focus groups, and/or narratives.45 In addition to identifying areas for improvement, a rigorous, ongoing monitoring and assessment program that demonstrates sustained activity and positive impact can also serve as a justification for continued or future funding.

The sponsor of a Web-based CoP on patient registries, in collaboration with the governing body, should develop and implement a plan for monitoring and assessing the impact and health of the CoP. Examples of possible measures may include:

  • Activity
    • Number of visitors per month, number of new members per month, number of active members, number of new discussion threads started per month, number of members attending Webinars hosted by the CoP
  • Impact: relevance, usability and value generation
    • Perceived value as a resource for work, perceived role of the CoP as a provider of solutions, ease of finding relevant content, improvements in efficiency when setting up or operating a patient registry

Table 5. Summary of recommendations for management
  Recommendations
Characteristics
  • Structure access to the Web site such that some areas are visible to the general public (e.g., resources, blog posts and comments), and some areas or functions are only available to members (e.g., submitting comments to blog posts).
  • Implement a member registration system that collects basic contact information. Anonymous participation should not be permitted. Do not limit member registration (e.g., by total number of members or by any member characteristic such as education level, employer, or specialty), with the possible exception of a level of monitoring to reduce “spam” member registrations.
  • Target the Web-based CoP to a broad audience of registry participants; registry designers, managers, and operators; providers and clinicians; researchers; industry and registry sponsors; and payers.
  • Allow members to drive the topical direction of the CoP. Allow members to initiate new topics in discussion forums and participate in the decision of which topics will be the focus of Webinars, teleconferences, and other resources. Accommodate content topics that are international in nature, especially if they are suggested by members.
  • Create a marketing plan that takes into account typical use cases of the CoP, and allocate resources to support the plan. If more resources are available, consider strategies such as social media, proactive outreach to professional organizations and patient groups, and/or search engine optimization. Present the CoP as a tool for mutual engagement and reputation building.
  • Create and implement a monitoring and assessment plan that measures the activity taking place in the CoP and the impact of the CoP, as perceived by its users.
 

Feasibility

The development of a Web-based CoP on patient registries is feasible from operational, regulatory, and technical perspectives, as described in the Recommendations section of this report. Many examples of successful CoPs exist in the public health and other arenas, and these will be important resources for launching a Web-based CoP on patient registries. Two decisions will most affect the creation, operation, and feasibility of a Web-based CoP: the choice of sponsor and the scope of purpose.

If AHRQ or another Federal agency funds or sponsors the CoP, its activities will be subject to more regulations than if the CoP is sponsored by a private entity or other nongovernment agency. While these considerations are important and are covered in more detail in the Considerations for a Federal Sponsor section, they are not sufficiently difficult that they represent insurmountable barriers to hosting a federally-sponsored Web-based CoP. The recently-formed Federal Virtual Community of Practice group will be an invaluable resource in learning from the experience of other Federal agencies and collaboratively working through issues that arise in hosting a Web-based CoP.

The second decision point which could most profoundly shape the Web-based CoP is the stated purpose and scope of the CoP. Conceivably, if the CoP is intended to address a broad range of topics and serve multiple different types of stakeholders with varying levels of familiarity with patient registries, more resources could be required than if the purpose were more narrowly focused. However, as suggested in the Topical Scope section of this report, if the sponsor allows the focus of the CoP content to be driven by the members themselves, members will be more engaged and more likely to actively participate, perhaps alleviating some of the administrative burden of CoP staff (e.g., spurring conversation on a topic by asking questions or prompting discussion). The goal of a CoP that is responsive to the specific interests of stakeholders, no matter how broad the topic, and not merely generating content for its own sake, is therefore feasible.

Cost Estimate

Based on the information available from the experience of existing online CoPs and the recommendations provided in this report, it is possible to propose a general cost estimate for the tasks associated with launching and managing a Web-based CoP on patient registries. Chronologically, these tasks generally fall into the categories of one-time setup tasks and ongoing annual operations and maintenance tasks. Additionally, many tasks ongoing for the life of the Web-based CoP can be implemented along a continuum with increasing amounts of resources invested. In Table 6, this continuum is represented by broad categories of “basic,” “intermediate,” and “advanced” models.

Setting up a Web-based CoP would involve establishing a governing body, writing a charter, articulating a communication plan, developing policies and procedures, determining functional requirements for the Web site, and marketing the CoP. The sponsor would need to select a technology vendor and work with them to design, configure, and launch the Web site. Depending on the level of additional stakeholder engagement and/or pilot testing the sponsor desires, these one-time costs are estimated to range from $150,000 to $500,000.

Annual operations and maintenance could involve hosting meetings for the governing body, administration and moderation of the CoP, managing and creating content (e.g., documents posted as resources, Webinars, teleconferences, case examples, “best practice” descriptions, templates, and other tools related to development and use of registries), marketing the CoP, providing help desk support for members, and monitoring and assessing the impact of the program. Table 6 displays the variation in resource investment that each one of these tasks could represent. Stakeholders indicated a strong interest in having more frequent Webinars and facilitated discussions, as reflected in the intermediate and advanced options.

Note that the estimates presented here do not include assumptions about the level of technical security and maintenance, reporting, or oversight that may be required if the Web-based forum is subject to Federal regulations (i.e., if AHRQ or another Federal agency is its sponsor—see Appendix F). These would incur an additional cost. It is also worth reiterating that while direct funding from AHRQ is one option for financing a Web-based CoP on patient registries, other funding options may exist. Once selected, the steering committee or other governing body may be a valuable resource in identifying alternate financing options, including a public-private partnership.

Table 6. Estimated costs for annual operations of a Web-based CoP on patient registries
Assumptions Basic Intermediate Advanced
*Costs are highly dependent on the level of hands-on support invested by the sponsor, and whether the Web-based CoP is subject to Federal regulations regarding reporting and accessibility.
Setup Cost = $100,000 to $200,000
  • Establish governing body, write charter, develop operational guidelines
  • Off-the-shelf technology platform, no customizations
  • No additional stakeholder engagement for requirements or pilot testing of Web site
Cost = $200,000 to $400,000
  • Establish governing body, write charter, develop operational guidelines
  • Off-the-shelf technology platform, some customizations
  • Additional stakeholder engagement for requirements, no pilot testing of Web site
Cost = $400,000 to $600,000
  • Establish governing body, write charter, develop operational guidelines
  • Customized off-the-shelf technology platform
  • Additional stakeholder engagement for requirements and pilot testing of Web site
Annual Operations* Cost = $50,000 to $100,000
  • Governing body meets remotely every quarter
  • Automated member enrollment
  • Monthly moderating of discussion forums
  • Monthly assistance for members forming working groups
  • Weekly email help desk support
  • Produce content (e.g., case examples, templates or toolkits) annually
  • Maintain news and events calendar that requires staff approval
  • Maintain resource library, seek out additional materials
  • Quarterly Webinars
  • Open-ended member satisfaction surveys
  • Regular outgoing emails for marketing
Cost = $200,000 to $400,000
  • Governing body meets remotely every quarter
  • Automated member enrollment
  • Monthly moderating of discussion forums
  • Monthly assistance for members forming working groups
  • Weekly email help desk support
  • Produce content (e.g., case examples, templates or toolkits) annually
  • Maintain news and events calendar that requires staff approval
  • Maintain resource library, seek out additional materials
  • Quarterly Webinars
  • Open-ended member satisfaction surveys
  • Regular outgoing emails for marketing
Cost = $400,000 to $600,000
  • Governing body meets every quarter (3 remote meetings and 1 in-person meeting)
  • Review members enrolled to verify affiliations and conflicts of interest
  • Daily moderating of discussion forums
  • Regular assistance for members forming working groups
  • Daily email help desk support
  • Produce content (e.g., case examples, templates or toolkits) quarterly
  • Seek out news and events to add to calendar
  • Maintain resource library, seek out additional materials, and develop indexing system
  • Monthly Webinars
  • Focus group discussions to assess CoP impact and health
  • Hire professional marketing firm

Next Steps

Should AHRQ decide to fund a Web-based CoP on patient registries, the Agency should consider taking the following four immediate next steps.

  1. Determine the scope and target audience for a Web-based CoP on patient registries, using the principles and recommendations described in this report. The scope should specify the range of topics that will be covered in the CoP, including whether the CoP will focus exclusively on issues relevant to U.S. registries or address issues relevant to both U.S. and international registries. AHRQ should also define the primary target audience, secondary audience, and minimum duration (e.g., 3 years) needed to attract initial members.
  2. Determine the level of internal and external resources available for this initiative. Internally, AHRQ should determine to what extent the Web-based CoP will be funded and sponsored by AHRQ. Resources should be allocated either at AHRQ or at its designee to allow for sufficient governance, management, and technical maintenance of the Web-based CoP, as described above. If it is decided that the CoP will not be funded and sponsored by AHRQ, AHRQ can begin to explore external stakeholder groups or partners that may be willing to sponsor such a program. As a thought leader in the patient registries field, AHRQ has much to offer the sponsor of a CoP and should seek to collaborate closely with the sponsor of such an initiative.
  3. Establish a stakeholder steering committee. Before work begins on developing the Web-based CoP, the sponsor should establish a stakeholder steering committee that will help guide the development, launch, and ongoing management of the CoP. Early and continuous stakeholder involvement will ensure that the Web-based CoP is a tool that will be useful and relevant to stakeholders.
  4. Determine the functional requirements for the technology platform needed to carry out the program goals and objectives. AHRQ or its designee should explore the information technology and Web hosting vendors available for hosting the Web-based CoP. Because the Web presence of the CoP will be the primary way that members interact with the community, the Web site plays a crucial role in facilitating and encouraging member participation. The vendor should be template- and/or module-driven, as described above, and should offer the flexibility to accommodate many different types of stakeholders and their topics of interest.

Conclusions

The background research and stakeholder input summarized in this report suggest that there is a very strong interest in a Web-based CoP on patient registries, and that it is a practical, feasible, and timely idea. Stakeholders have a broad range of questions about creating and using registries. Researchers, health care providers, and representatives from government, industry, payer, and patient organizations all noted that the value of a Web-based CoP would depend on its ability to remain relevant and enable them to do their work better, faster, or more easily. Stakeholders also saw the value in a tool that would facilitate networking, interaction, and collaboration; improve efficiencies in resource use and reduce duplication of effort; improve patient care and outcomes; provide an organized system for learning and information sharing; and serve as a collective voice for patient registries on the national stage.

The proposed Web-based CoP on patient registries would serve as a widely available tool to centralize, cultivate, and facilitate the interactions that are currently taking place within the patient registries community. The CoP would help facilitate broader sharing of information and further enhance existing knowledge exchange networks by providing an online mechanism for stakeholders to engage in ongoing discussions about registry science and methodology, identify and debate emerging issues, and share challenges and successes from their own experience. Collaboration with existing networks or initiatives and leveraging existing resources on registry best practices will be important for the success of the CoP.

The software required to conduct such a Web-based CoP is readily available "off-the-shelf" so that the technical focus would be on selecting which features to use and apply rather than building custom software. The U.S. Department of Health and Human Services and other Federal agencies already sponsor numerous Web-based CoPs, providing precedents regarding regulatory and reporting requirements of such a Federally sponsored tool. Furthermore, stakeholders interested in patient registries see AHRQ as a logical choice for advancing this effort, given its reputation as an unbiased, knowledgeable, and trusted leader in the field of observational research methods, quality improvement, and patient registries.

Stakeholders provided concrete feedback about the features they would like to see in a Web-based CoP on patient registries. In keeping with the vision laid out by stakeholders, a Web-based CoP on patient registries should be established that would provide a mechanism for persons interested in starting registries, or using existing ones, to engage in discussions with their peers and other subject matter experts about a broad range of registry-related topics.

Should AHRQ decide to move forward with a Web-based CoP on patient registries, the agency should determine the scope and target audience for the CoP; determine the level of internal and external resources available for this initiative; establish a stakeholder steering committee; and determine the functional requirements for the technology platform. As the CoP is launched, continued discussions should be held with stakeholders to further refine requirements and priorities, identify opportunities for collaboration, and to ensure that the CoP best serves its intended audience.

References

  1. Appendix 2 Gliklich RE, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 2nd ed. (Prepared by Outcome DEcIDE Center [Outcome Sciences, Inc. d/b/a Outcome] under Contract No. HHSA290200500351 TO3.) AHRQ Publication No. 10-EHC049. Rockville, MD: Agency for Healthcare Research and Quality. September 2010. p. 9.
  2. Gliklich RE, Dreyer NA, eds. Registries for Evaluating Patient Outcomes: A User's Guide. 2nd ed. (Prepared by Outcome DEcIDE Center [Outcome Sciences, Inc. d/b/a Outcome] under Contract No. HHSA290200500351 TO3.) AHRQ Publication No. 10-EHC049. Rockville, MD: Agency for Healthcare Research and Quality. September 2010.
  3. Patient-Centered Outcomes Research Institute. "About Us." www.pcori.org/about. Accessed August 29, 2013.
  4. PCORI Methodology Standards. December 14, 2012. http://pcori.org/assets/PCORI-Methodology-Standards.pdf. Accessed August 29, 2013.
  5. Agency for Healthcare Research and Quality. "Developing a Registry of Patient Registries (RoPR)." http://effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=690. Accessed August 29, 2013.
  6. Centers for Disease Control and Prevention. "Welcome to the Communities of Practice Resource Kit!" www.cdc.gov/phcommunities/resourcekit/index.html. Accessed August 29, 2013.
  7. Wenger E, McDermott R, Snyder WM. Cultivating Communities of Practice. Boston: Harvard Business School Press; 2002. p. 4.
  8. Agency for Healthcare Research and Quality. "Practice-Based Research Networks (PBRNs)." http://pbrn.ahrq.gov. Accessed August 29, 2013.
  9. Wenger E, Trayner B. Personal e-mail communication. 21 September 2012.
  10. Wenger E. "Slide: Levels of participation." http://wenger-trayner.com/resources/slide-forms-of-participation. Accessed August 29, 2013.
  11. Wenger E, Trayner B. "What are three key success factors for communities of practice?" http://wenger-trayner.com/resources/key-success-factors. Accessed August 29, 2013.
  12. Hara N, Hew KF. Knowledge-sharing in an online community of health-care professionals. Information Technology & People. 2007.;20(3):235-61.
  13. Brooks F, Scott P. Exploring knowledge work and leadership in online midwifery communication. J Adv Nurs. 2006 Aug;55(4):510-20.
  14. Global Alliance for Nursing and Midwifery Communities of Practice (GANM). "Working Together for Health Knowledge Exchange." http://knowledge-gateway.org/ganm. Accessed August 29, 2013.
  15. Valaitis RK, Akhtar-Danesh N, Brooks, et al. Online communities of practice as a communication resource for community health nurses working with homeless persons. J Adv Nurs. 2011 Jun;67(6):1273-84.
  16. Thomas AU, Fried GP, Johnson P, et al. Sharing best practices through online communities of practice: a case study. Hum Resour Health. 2010 Nov 12;8:25.
  17. Hoffmann T, Desha L, Verrall K. Evaluating an online occupational therapy community of practice and its role in supporting occupational therapy practice. Aust Occup Ther J. 2011 Oct;58(5):337-45.
  18. Greene EA, Griffin AS, Whittle J, et al. Development and usage of eXtension's HorseQuest: an online resource. J Anim Sci. 2010;88:2829-37.
  19. OrthoMind LLC. "About Us." www.orthomind.com/AboutUs. Accessed August 29, 2013.
  20. Falkman F, Fustafsson M, Jontell M, et al. SOMWeb: a semantic Web-based system for supporting collaboration of distributed medical communities of practice. J Med Internet Res. 2008 Jul-Sep;10(3):e25.
  21. Meessen B, Kouanda S, Musango L, et al. Communities of practice: the missing link for knowledge management on implementation issues in low-income countries? Trop Med Int Health. 2011 Aug;16(8):1007-1014.
  22. Ekberg J, Ericson L, Timpka T, et al. Web 2.0 systems supporting childhood chronic disease management: design guidelines based on information behavior and social learning theories. J Med Syst. 2010;34:107-17.
  23. Curran JA, Murphy AL, Abidi SSR, et al. Bridging the gap: knowledge seeking and sharing in a virtual community of emergency practice. Eval Health Prof. 2009 Sep;32(3):312-25.
  24. Barnett SR, Jones SC, Bennett S, et al. General practice training and virtual communities of practice – a review of the literature. BMC Fam Pract. 2012 Aug 21;13(1):87. [Epub ahead of print]
  25. Cassidy L. Online communities of practice to support collaborative mental health practice in rural areas. Issues Ment Health Nurs. 2011;32(2):98-107.
  26. National Institutes of Health. National Cancer Institute. "About Research to Reality." https://researchtoreality.cancer.gov/about. Accessed August 29, 2013.
  27. Michele Rovins. Personal telephone communication. 2 August 2012.
  28. National Association of State Directors of Special Education, Inc. "IDEA Partnership." http://nasdse.org/Projects/IDEAPartnership/tabid/413/Default.aspx. Accessed August 29, 2013.
  29. Joanne Cashman, Patrice Linehan, Mariola Rosser. Personal telephone communication. 3 August 2012.
  30. Centers for Disease Control and Prevention. "Welcome to the Communities of Practice Resource Kit!" www.cdc.gov/phcommunities/resourcekit/index.html. Accessed 14 August 2012.
  31. Alissa Gallagher. Personal e-mail communication. 24 July 2012.
  32. Physician Consortium for Performance Improvement. PCPI Orientation, April 5, 2013. www.ama-assn.org/resources/doc/cqi/pcpi-april2013-orientation.pdf. Accessed August 29, 2013.
  33. Richardson, Dana. National Quality Registry Network. October 12, 2012. www.acrinformatics.org/LinkClick.aspx?fileticket=slORGgcNSME%3D&tabid=140. Accessed April 25, 2014. Slide 12.
  34. Wenger E, Trayner B. "Knowledge is power?" http://wenger-trayner.com/resources/knowledge-is-power. Accessed August 29, 2013.
  35. Agency for Healthcare Research and Quality. "AHRQ At A Glance." www.ahrq.gov/about/ataglance.htm. Accessed 5 September 2012.
  36. Rajendra Babu H, Gopalaswamy M. Use of Web 2.0 tools and technologies for science communication in biomedical sciences: A special reference to blogs. Int J Library Information Science. 2011 May;3(5):85-91.
  37. Michele Rovins. Personal telephone communication. 2 August 2012.
  38. Joanne Cashman, Patrice Linehan, Mariola Rosser. Personal telephone communication. 3 August 2012.
  39. Margaret Farrell. Personal e-mail communication. 25 June 2012.
  40. Gemmen EK, Pashos CL, Blanchette CM, et al. The role of patient registries in evidence development: similarities and differences between Europe and North America. Presented at ISPOR EU Workshop, 30 November 2009.
  41. "Disease Registries in Europe." Orphanet Report Series, Rare Diseases collection, January 2011. www.orpha.net/orphacom/cahiers/docs/GB/Registries.pdf. Accessed August 29, 2013.
  42. McLure Wasko M, Faraj S. "It is what one does": why people participate and help others in electronic communities of practice. Journal of Strategic Information Systems. 2000;9(2):155-73.
  43. Wenger E, Trayner B. "Knowledge is power?" http://wenger-trayner.com/resources/knowledge-is-power. Accessed August 29, 2013.
  44. Castro MC. The Macuarium Set of Community of Practice Measures. First edition, March 2006.
  45. Wenger E, Trayner B, de Laat M. Promoting and Assessing Value Creation in Communities and Networks: A Conceptual Framework. Heerlen, Netherlands:Ruud de Moor Centrum; 2011. http://wenger-trayner.com/documents/Wenger_Trayner_DeLaat_Value_creation.pdf. Accessed August 29, 2013.

Appendixes

Appendix A. Glossary of Terms

The definitions below are provided in order to clarify how these terms are used in this report and are not intended to represent the full and accurate definitions of these terms as they may appear in other publications or Web sites.

Blog—Derived from the previously used term “web log.” A blog is “a discussion or information site published on the World Wide Web consisting of discrete entries (“posts”) typically displayed in reverse chronological order so the most recent post appears first.”1 Blogs can be authored by a single individual or a large group of individuals (often called “multi-author blogs” or MABs), and can be limited or broad in topical focus.

Chat room—A form of synchronous, online text-based conferencing which enables multiple users to exchange text messages that appear to all users in the chat room simultaneously. The key aspect of a chat forum is that the interaction between the users happens in real time. The text discussions from a chat room session are not typically archived after the session is complete, although they can be. Some forms of chat rooms also include the ability to use voice in addition to text.2

Community of Practice (CoP)—A group of people who share a concern or passion for something they do and learn how to do it better as they interact regularly.3

Discussion forum or message board—A Web application that allows users to hold conversations in the form of posted messages. These messages are often asynchronous in nature and archived for future viewing. Messages may need to be approved by a moderator or administrator before they become visible in the forum. Discussion forums can be subdivided by conversation topic into multiple “subforums.” Each new discussion started in a forum is sometimes called a “thread.”4

Mashup—Any Web application that combines multiple Web 2.0 functions from different sources to establish a new service.

Member directory—Stores and displays information about community members in a roster format. The information is typically provided by members themselves when they register to become part of the community, and can include basic contact information (name and email address) or more detailed information such as role (e.g., researcher, clinician, patient). Some information not explicitly provided by members (e.g., date joined, level of activity on the site) can also be displayed here for administrative use.

Message board—See Discussion forum.

Microblog—A blog that allows only very brief blog posts, typically with a character limit of 150–200 or less. Small images may be included as well as brief audio and video clips. The most popular microblog is Twitter, although sites like Facebook and LinkedIn also facilitate microblogging in the form of “status updates.”

Web-based (or online, virtual) Community of Practice—A CoP that is developed and/or maintained using the Internet.5

Podcast—A multimedia digital file (audio, video, or both) made available for free or for purchase on the Internet. Users download the podcast to a portable media player or computer to listen or view.6

Rating or ranking—A functionality of a Web application that allows users to assign a value to an item that reflects their positive (and sometimes negative) response to that item. The value attribution can be mono-directional (e.g., a “thumbs up” option that the user can choose to select or not select), bi-directional (e.g., both “thumbs up” and “thumbs down” options) or continuous (e.g., 1–5 stars). Value can be attributed to blog posts, posts in discussion forums, and almost any other content hosted on a Web site.

Resource library—A designated area of a Web site that houses informational resources in an organized manner. The resources can include documents, files, citations of external publications, and hyperlinks to external Web sites. They can be organized by topic, source, or other characteristic, and may or may not be searchable for users.

Rich Site Summary (RSS)—A Web feed format used to publish frequently-updated works (such as blog entries, news headlines, etc.) in a consolidated, standardized way for the viewer. An RSS document (called a “feed”) includes full or summarized text, plus metadata such as publishing dates and authorship. Users typically subscribe to a feed, and then view the feed with Web-based, desktop-based, or mobile-device-based software called an “RSS reader.”7

Social media—A group of Internet-based applications that build on the ideological and technological foundations of Web 2.0, and that allow the creation and exchange of user-generated content.8

Tag—A nonhierarchical keyword assigned to a piece of information such as a blog post, digital image, or computer file. This kind of metadata helps describe an item and allows it to be found again by browsing or searching. Tags are usually single words or very short phrases, and can be assigned by the item’s creator or by its viewer.9

Tag cloud—A visual representation of text data, typically used to depict tags on Web sites. The importance of each tag is usually shown with font size or color. This format is useful for quickly perceiving the most prominent terms and for locating a term alphabetically to determine its relative prominence. When used as Web site navigation aids, the terms can be hyperlinked to items associated with the tag.10

User-generated content (UGC)—Various forms of media content that are publicly available on the Internet and created by end-users. To be considered UGC, the content must be published on either a publicly accessible Web site or on a social networking site accessible to a selected group of people (i.e., not emails or private messages), must show a certain amount of creative effort (i.e., not replication of existing content), and must have been created outside of professional routines and practices (i.e., not created with a commercial market context in mind).11

Web 2.0—Web applications that facilitate interactive information sharing, interoperability, user-centered design, and collaboration on the World Wide Web.12

Webinar or Webconference—A synchronous, online video and audio conferencing event. Typically, a Webinar consists of one or more presenters and multiple attendees or audience members. The video portion of the Webinar may display the presenter themselves as they address the audience, or their computer screen (e.g., as they display a slide presentation).

Wiki—A Web site developed collaboratively by a community of users, allowing any user to add and edit content.13 Wikis may serve different purposes, including knowledge management or notetaking. A single page in a wiki Web site is referred to as a “wiki page,” while the entire collection of pages, which are usually interconnected by hyperlinks, is “the wiki.”14

Appendix B. Organizations Represented at Stakeholder Meeting

Government
  • Agency for Healthcare Research and Quality (AHRQ)
  • Agency for Toxic Substances and Disease Registry (CDC)
  • Center for Devices and Radiological Health (FDA)
  • Knowledge Management Office (CDC)
  • National Cancer Institute (NIH)
  • National Center on Birth Defects and Developmental Disabilities (CDC)
  • National Heart Lung and Blood Institute (NIH)
  • National Institute of Child Health and Human Development (NIH)
  • National Institute of Dental and Craniofacial Research (NIH)
  • National Institute of Diabetes and Digestive and Kidney Diseases (NIH)
  • National Library of Medicine (NIH)
  • Office of Rare Diseases Research (NIH)
Industry
  • AMAG Pharmaceuticals
  • Celgene
  • Ground Zero Software
  • IDEA Partnership
  • Intellica Corporation
  • Intellicure Inc.
  • Janssen, LLC
  • KAI Research
  • Sanofi Pasteur
  • Synageva BioPharma
Patient
  • American Association for the Advancement of Science (AAAS)
  • American Thrombosis & Hemostasis Network
  • Cystic Fibrosis Foundation
  • National Foundation for Celiac Awareness
  • National Psoriasis Foundation
  • RASopathies Network USA
  • VHL Family Alliance
Payer
  • America’s Health Insurance Plans
  • Centers for Medicare & Medicaid Services (CMS)
  • Humana
  • Kaiser Permanente
  • UnitedHealth Group
  • Wellpoint
Provider
  • American Academy of Family Physicians
  • American Association on Intellectual and Developmental Disabilities (AAIDD)
  • American College of Gastroenterology
  • American College of Rheumatology
  • American Medical Association
  • American Society of Breast Surgeons
  • American Society of Plastic Surgeons
  • Avalere Health, LLC
  • Children’s Interstitial and Diffuse Lung Disease (chILD) Research Network
  • North American Association of Central Cancer Registries
  • Practice Transformation Institute
  • Society of Thoracic Surgeons
  • University of Kansas Medical Center
Researcher
  • American Heart Association
  • American Institutes for Research
  • Boston University School of Public Health
  • Center for Medical Technology Policy (CMTP)
  • Children’s Hospital of Philadelphia
  • Consortium of Rheumatology Researchers of North America (CORRONA)
  • Duke University Medical Center
  • Feinstein Institute, North Shore-Long Island Jewish Health System
  • George Washington University
  • Massachusetts General Hospital
  • National Comprehensive Cancer Network
  • Portland VA Medical Center
  • Radiation Oncology Institute
  • University of Alabama Birmingham
  • UCSD Department of Pediatrics
  • UCSF Clinical & Translational Science Institute
  • University of New Mexico
  • University of Pittsburgh Medical Center
  • University of Washington

Appendix C. Stakeholder Meeting Agenda

Developing a Web-Based, Collaborative Forum on Patient Registries
Exploring the potential value for encouraging useful discussion, sharing of best practices, and debate on new challenges
Stakeholder meeting sponsored by the Agency for Healthcare Research and Quality
Monday, March 26, 2012
Holiday Inn National Airport
2650 Jefferson Davis Highway, Arlington, Virginia 22202

Meeting Agenda
Time Event
9:00-9:30 am Registration and Breakfast
9:30-9:45 am Welcome and Meeting Overview
Presenters:
  • Richard Gliklich, Outcome DEcIDE Center
  • Elise Berliner, Agency for Healthcare Research and Quality
Session Objectives:
  • Describe project purpose
  • Review meeting objectives and agenda
9:45-10:25 am Introduction to Web-based Forums and Communities of Practice
Presenters:
  • Daniel Campion, Outcome DEcIDE Center
  • Mamie Jennings Mabery, Centers for Disease Control and Prevention
Session Objectives:
  • Describe the concept of communities of practice (CoP) and role of a Web-based forum within those communities
  • Outline the possible components of a Web-based forum
  • Describe the CDC’s initiative to promote the development of CoPs and use of Web-based tools
10:15-11:15 am Need for a Web-based Registries Forum
Session Objective: Participants will discuss the following questions in small groups. Each group will then report out to the larger group.
  • How do you currently learn about registry practices and interact with others regarding this subject?
  • Is there a need for a Web-based forum on patient registries?
  • If so, what would you most like to get out of such a program?
  • What reservations, if any, would you have about participating in such a forum?
  • What topics about registries would you like see addressed through the forum?
11:15-11:30 am Break
11:30 am-12:30 pm Lessons from Existing Web-based Forums
Presenters:
  • Margaret Farrell, Research to Reality Program, National Cancer Institute
  • Joanne Cashman, IDEA Partnership, sponsored by the Office of Special Education Programs (OSEP), US Department of Education
Session Objective: Representatives from several existing programs will describe their communities of practice and use of Web-based tools.
  • Program purpose
  • Target audience
  • Key features/services
  • Infrastructure: technology, human resources, funding
  • Metrics for measuring success
  • Lessons and suggestions for a potential registries forum
  Working Lunch
12:30-3:00 pm Key Considerations for a Web-based Forum on Patient Registries
Session Objective: Participants will work in small groups to outline key components for a potential collaborative forum on patient registries and then report out to the larger group for discussion.
12:30-1:30 pm
  • Purpose and goals
  • Participants and intended audience
1:30-2:10 pm
  • Specific services and/or features
  • Technology platform
2:10-2:40 pm
  • Governance and management
  • Promotion
2:40-3:00 pm
General comments and discussion of cross-cutting themes and critical design issues that may be emerging during discussion
3:00-3:15 pm Break
3:15-3:45 pm Value proposition and funding
Session Objective: Discuss the following questions.
  • Which of the possible goals of the forum would be the highest priority for you? Why?
  • Is it reasonable to think that you might spend a significant amount of time (>1 hour/month) engaged in an on-line dialogue with others through the forum? Under what conditions might you feel comfortable participating at this level?
  • What could be potential sources of support for such a forum?
3:45-4:00 pm Next Steps and Concluding Comments

Appendix D. Screen Shots of Research to Reality Web Site

Figure D-1. Research to Reality home page15

Figure D-1 is a screen shot of the home page for the Research to Reality (R2R) Web site.  From this page you can see tabs to navigate to other areas of the Web site, including Discussions, Cyber-Seminars, Featured R2R Partners, Mentorship, Events, About, Sign Up, and Login.  This screen shot from the Home page was taken on 25 September 2012 and includes an announcement for an upcoming cyber-seminar (webinar), a recent activity area, small events calendar, and a link to a video tutorial about Research to Reality.

Figure D-2. Research to Reality discussions page16

Figure D-2 is a screen shot of the Discussions Page on the R2R Web site.  It shows a vertical list of ongoing discussion topics, which users can click on to view individual conversations.  Examples of topics include Exercise and Response Rates and How can social media enhance community-based prevention programs?

Figure D-3. Research to Reality cyber-seminars page17

Figure D-3 is a screen shot of the Cyber-Seminars page on the R2R Web site.  This screen shot was taken on 25 September 2012, and the page features a description of the next upcoming cyber-seminar, on the topic of Implementing evidence-based interventions within healthcare system: experiences from the R2R mentorship program.  Below this is a chronological list of past cyber-seminars, with links to archived recordings.

Appendix E. Screen Shots of TAcommunities Web Site

Figure E-1. TAcommunities home page18

Figure E-1 is a screenshot of the TAcommunities Home Page.  From this page you can see tabs to navigate to other areas of the Web site, including Communities, People, Discussions, Documents, Suggest a New Community, Links, and Help.  The Home page features a login area for members, a link to join the community, and a brief description of the CoP's purpose and structure.

Figure E-2. TAcommunities communities page19

Figure E-2 is a screen shot of the Communities Page on the TAcommunities Web site.  It depicts a vertical list of communities that exist in the CoP by topic, including Dissemination, Fiscal Accountability, and Exiting.

Figure E-3. TAcommunities documents page20

Figure E-3 is a screen shot of the Documents page on the TAcommunities Web site.  It shows a folder structure for the documents stored in this area; the folders are organized by community and topic.

Figure E-4. TAcommunities links page21

Figure E-4 is a screen shot of the Links page on the TAcommunities Web site.  It shows a list of links to external resources and Web sites on topics relevant to the members of the TAcommunities CoP.

Appendix F. Considerations for a Federal Sponsor

OMB Clearance Under the Paperwork Reduction Act

The Paperwork Reduction Act of 1995 (PRA) requires that the Office of Management and Budget (OMB) approve any standardized information collection by a Federal agency which is administered to ten or more people within a 12-month period.22 Any Web site managed or sponsored by a Federal agency must assess the extent to which their activities fall under the PRA and need to be approved by OMB.

Existing Web-based CoPs have approached this issue in different ways, depending upon their individual situation. For example, the Web-based CoPs for the TA&D Network (www.tacommunities.org and www.tadnet.ning.com) exist primarily to disseminate information, rather than collect it. They have therefore determined that they are not eligible for OMB clearance and have not sought OMB clearance under the PRA.23

In the case of an interactive, Web-based CoP as is described in this report, this assessment is complicated by the potential use of newer technologies that have been developed since the PRA was originally written. Recent communication from OMB has clarified the extent to which the PRA relates to these new technologies. In the April 7, 2010 memorandum, “Social Media, Web-based Interactive Technologies, and the Paperwork Reduction Act” (attached here as Appendix G), OMB identifies technologies and Web-based activities that have come into frequent use by Federal agencies, and clarifies which of these fall under PRA. While AHRQ should analyze this document in its entirety to understand its full context, Table 7 summarizes some highlights from the memo that may be pertinent to the types of activities discussed in this report.

Section 508

AHRQ should assess the extent to which any Web-based COP will need to be compliant with Section 508 of the Rehabilitation Act of 1973, which requires that Federal agencies’ electronic and information technology is accessible to people with disabilities.24 As noted in the footer on their Web site, www.tacommunities.org is compliant with Section 508,25 and has been since its inception.26

Freedom of Information Act

AHRQ should consider and prepare for the possibility that information made available on a Web-based CoP could lead to a request from the public under the Freedom of Information Act (FOIA) of 1966. Under this Act, “any person has a right, enforceable in court, to obtain access to Federal agency records, except to the extent that such records (or portions of them) are protected from public disclosure by one of nine exemptions or by one of three special law enforcement record exclusions.”27

It may be unlikely that AHRQ encounters such a request; www.tacommunities.org reported that they have not received a FOIA request to date. However unlikely such a request may be, it may be worth determining how AHRQ policy will apply to these situations, and the appropriate response of the Web-based CoP to such requests.

Privacy Provisions of the E-Government Act

AHRQ should assess to what extent a Web-based CoP will be subject to the privacy provisions of the E-Government Act of 2002. These provisions compel Federal agencies to conduct privacy impact assessments for electronic information systems and collections and make them publicly available, post privacy policies on agency Web sites used by the public, translate privacy policies into a standardized machine-readable format, and report annually to OMB on compliance with section 208 of the E-Government Act of 2002.28

The Research to Reality Web site is compliant with these privacy provisions and underwent a privacy impact assessment both through the U.S. Department of Health and Human Services and NCI.29 The privacy guidelines for Research to Reality are published publicly on their Web site, at https://researchtoreality.cancer.gov/about/policies.

Table F-1. Web-based activities that do and do not require OMB clearance under PRA30
OMB Clearance Web-based Activities
PRA = Paperwork Reduction Act
These activities do not fall under PRA
  • Use of wikis
  • Posting comments
  • Functions that allow users to rate, rank, vote on, flag, tag, label, or similarly assess the value of ideas, solutions, suggestions, questions, and comments posted by Web site users
  • Any general request for comments or feedback, including those that pose specific questions designed to elicit public feedback, as long as it is not presented in survey form and the responses are unstructured
    • This applies regardless of the format of the request for comments. For example, the request may be made via social media Web sites; blogs; photo, or video sharing Web sites; or online message boards
    • This also applies if the request takes the form of a contest (i.e., a prize will be given)
  • Posting an agency email address or using an application for brainstorming or idea-generating on its Web site to enable the public to submit feedback
  • Collecting email addresses for the purpose of sending agency updates, alerts, publications, or email subscription services
  • Collecting mobile phone numbers for the purpose of text notification lists
  • Collecting addresses for RSS feeds
  • Hosting a public meeting
  • The use of interactive meeting tools such as public conference calls, Webinars, blogs, discussion boards, forums, message boards, chat sessions, social networks, and online communities
  • Information collected to create user accounts or profiles for agency Web sites, including email address, username, password, and geographic location (e.g., State, region, or zip code)
  • Features that allow users to customize the appearance of an agency Web site (e.g., faceted navigation, filters)
  • Collecting Web site data to create a tag cloud
  • Collecting information necessary to complete a voluntary commercial transaction
These activities do fall under PRA
  • Distributing any type of survey (including Web polls and satisfaction surveys) that poses identical, specific questions. This applies regardless of format or mode of administration, including Web polls, satisfaction surveys, pop-up windows, those sent via an email list, during in-person meetings or focus groups.
  • Requesting information from respondents beyond name and email or mailing address (e.g., age, sex, race/ethnicity, employment, or citizenship status)
  • Use of a wiki to collect information that an agency would otherwise gather by asking for responses to identical questions (e.g., posting a spreadsheet into which respondents are directed to enter data)
  • Use of online accounts to collect information for programmatic purposes (e.g., using FAFSA to determine eligibility for student aid)

Appendix G. Office of Management and Budget Memo on Social Media, Web-Based Interactive Technologies, and Paperwork Reduction Act

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

This is a seven-page memorandum for the heads of executive departments and agencies and independent regulatory agencies about social media, Web-based interactive technologies, and the Paperwork Reduction Act. it can be read online in an accessible document at http://www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf

Appendix H. References for Appendixes A–G

  1. Wikipedia. “Blog.” http://en.wikipedia.org/wiki/Blog. Accessed August 29, 2013.
  2. Wikipedia. “Chat room.” http://en.wikipedia.org/wiki/Chat_room. Accessed 13 September 2012.
  3. Wenger E. “Communities of Practice: a Brief Introduction.” http://wenger-trayner.com/wp-content/uploads/2012/01/06-Brief-introduction-to-communities-of-practice.pdf. Accessed August 29, 2013.
  4. Wikipedia. “Internet forum.” http://en.wikipedia.org/wiki/Internet_forum. Accessed August 29, 2013.
  5. Wikipedia. “Online community of practice.” http://en.wikipedia.org/wiki/Online_community_of_practice. Accessed August 29, 2013.
  6. Google search. “Podcast definition.” www.google.com/search?q=podcast+definition&rls=com.microsoft:en-us:IE-Address&ie=UTF-8&oe=UTF-8&sourceid=ie7&rlz=1I7GGHP_enUS413. Accessed August 29, 2013.
  7. Wikipedia. “RSS.” http://en.wikipedia.org/wiki/RSS. Accessed August 29, 2013.
  8. Kaplan AM, Haenlein M. Users of the world, unite! The challenges and opportunities of social media. Business Horizons, 2010, Vol.53(1), pp.59-68
  9. Wikipedia. “Tag (metadata).” http://en.wikipedia.org/wiki/Tag_(metadata). Accessed August 29, 2013.
  10. Wikipedia. “Tag cloud.” http://en.wikipedia.org/wiki/Tag_cloud. Accessed August 29, 2013.
  11. Kaplan AM, Haenlein M. Users of the world, unite! The challenges and opportunities of social media. Business Horizons, 2010, Vol.53(1), pp.59-68
  12. Rajendra Babu H, Gopalaswamy M. Use of Web 2.0 tools and technologies for science communication in biomedical sciences: A special reference to blogs. Int J Library Information Science. 2011 May;3(5):85-91.
  13. Google search. “Wiki definition.” www.google.com/#hl=en&sugexp= les%3B&gs_nf=1&cp=9&gs_id=x&xhr=t&q=wiki+definition&pf=p&output=search&sclient= psy-ab&oq=wiki+defi&gs_l=&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&fp=dc3c98d427bf7980&biw= 1920&bih=985. Accessed August 29, 2013.
  14. Wikipedia. “Wiki.” http://en.wikipedia.org/wiki/Wiki. Accessed August 29, 2013.
  15. National Institutes of Health. National Cancer Institute. “Research to Reality – Home.” https://researchtoreality.cancer.gov/home. Screen shot taken on 25 September 2012. Used with permission.
  16. National Institutes of Health. National Cancer Institute. “Research to Reality – Discussions.” https://researchtoreality.cancer.gov/discussions. Screen shot taken on 25 September 2012. Used with permission.
  17. National Institutes of Health. National Cancer Institute. “Research to Reality – Cyber-Seminars.” https://researchtoreality.cancer.gov/cyber-seminars. Screen shot taken on 25 September 2012. Used with permission.
  18. AED. “TAcommunities – Home.” www.tacommunities.org. Screen shot taken on 25 September 2012. Used with permission.
  19. AED. “TAcommunities – Communities.” www.tacommunities.org/community. Screen shot taken on 25 September 2012. Used with permission.
  20. AED. “TAcommunities – Documents.” www.tacommunities.org/document. Screen shot taken on 25 September 2012. Used with permission.
  21. AED. “TAcommunities – Links.” www.tacommunities.org/links/list. Screen shot taken on 25 September 2012. Used with permission.
  22. 44 U.S.C. chapter 35; see 5 CFR Part 1320.
  23. Michele Rovins. Personal telephone communication. 2 August 2012.
  24. Section 508 of the Rehabilitation Act (29 U.S.C. 794d), as amended by the Workforce Investment Act of 1998 (P.L. 105-220), August 7, 1998. www.section508.gov/index.cfm?fuseAction=1998Amend. Accessed August 29, 2013.
  25. TAcommunities. www.tacommunities.org/. Accessed August 29, 2013.
  26. Michele Rovins. Personal telephone communication. 2 August 2012.
  27. U.S. Department of Justice. “What is FOIA?” www.foia.gov/about.html. Accessed August 29, 2013.
  28. Office of Management and Budget. Memorandum for Heads of Executive Departments and Agencies. M-03-22, “OMB Guidance for Implementing the Privacy Provisions of the E-Government Act of 2002.” www.whitehouse.gov/omb/memoranda_m03-22. Accessed August 29, 2013.
  29. Margaret Farrell. Personal e-mail communication. 25 June 2012.
  30. Office of Management and Budget. Memorandum for the Heads of Executive Departments and Agencies, and Independent Regulatory Agencies. “Social Media, Web-Based Interactive Technologies, and the Paperwork Reduction Act.” www.whitehouse.gov/sites/default/files/omb/assets/inforeg/SocialMediaGuidance_04072010.pdf. Accessed August 29, 2013.

Citation

Gliklich RE, Campion DM, Khurana L, Leavy MB, Karl J. Options for Developing a Web-Based Forum on Patient Registries. Effective Health Care Program Research Report No. 44. (Prepared by the Outcome DEcIDE Center, under Contract No. 290-2005-0035-I.) AHRQ Publication No. 14-EHC020-EF. Rockville, MD: Agency for Healthcare Research and Quality. May 2014.

Return to Top of Page