Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care

Back to "About This Webcast."

Innovations in Stakeholder Engagement Conference, AMA

Webcast Transcript, November 3, 2011

Robert Hicks, Business Process Development Director, AMA
Brian Eaggleston, Physician Outreach and Retention, AMA    

HICKS: I'm Bob Hicks. I'm business process development director for the AMA. Thank you for inviting us to this very interesting conference.

We've been invited because we just completed our 2011 AMA App Challenge that went out to our physician panel.
And so mid-2010, you know, especially around the launch of the iPad, I think that was April of 2010, I was tasked with starting to explore the development of mobile applications for our constituent panel, which are stakeholders, which are U.S. physicians.

Things we immediately began to see, like according to Manhattan Research, there was a rapid adoption of smartphone by physicians. Some 80 percent of all physicians have a smartphone. And over half of them were using an -- an iDevice, a -- a -- you know, an iPhone, an iPad of some sort.

We were looking to develop a few of our own apps. We have a lot of content but, you know, don't necessarily know how to bundle that out there.

And plus, we saw that there were a number of apps already out there on iPhone but not very many seemed to be focused specifically for the physician.

So put together one of my passions and that is the crowd sourcing, collaborative platform that, you know, Ericand Lakshmi were talking about and came up with the idea of doing an app contest.

And that is an app challenge where we actually went out and went to our physicians, to our stakeholder panel, and asked them, you know, submit your idea. What -- what's your idea for a iPhone app, a smartphone app, that will, you know, help you in your daily practice.

And we opened it to all physicians, resident fellows and student members, medical students, whether you were a member or not. We built a micro-site for the challenge. And then eventually we, you know, our members would vote on -- our members would vote on the winning apps that were submitted, or the apps that were submitted.

And we were gonna choose two winners, one physician and one from what we were calling the physician in training, so student, resident, fellow.

EAGGLESTON: So with launching this, we really did not know what to expect at all. We had -- we set our goals low and high, hoping we fell somewhere in there.

So we were looking for 2 to 500 ideas that were useful. And again, it was really trying to engage both our member audience and our non-member audience. Obviously, as you know, the physicians audience is not easy to get in front of, especially if you're not dying.

So this is a very, you know, a fun, casual way that we were trying to engage with them. And I -- also too, I'll hopefully be able to get at least one or two ideas that we truly could end up developing and also just to bring a new positive light to the AMA.

We, around the time that we were brainstorming on this, was in the heat of a health system reform so I was looking for any campaign to work on, other than something around health system reform, as you can imagine.


So a little bit on the timeline here, we -- there was mention in the earlier presentations about how -- how to keep things, how long to keep things open so that they don't die out. That was also definitely a concern of ours.

It was two-part. We didn't want it to die out but we also knew how busy this particular audience was. So we struggled with that a little bit.

But we had -- entries were open for about three months. Then we sifted through the entries, 300 and some -- we'll get to that in a minute -- so to weed down to about 10 that we could present up to our members.

Members voted for roughly around six weeks. And then, as you can see from this calendar we are in the preparation of who the two winners are.

And then, on November 14, we have our huge inner meeting in New Orleans for our house of delegates and that is where we are presenting the two app challenge winners.

(HICKS): So we know who the winners are but we just can't share at this time, so...

EAGGLESTON: So I'll briefly go over just some of the tactics, marketing tactics that we used. Again, this audience, hard to get to, so we needed to be able to find the audience that was interested in this, engage them, and also support them through the process.

So we used all the traditional marketing tactics. Again, we launched up our little micro-site separate from the AMA site. We also, the actual URL for this is



(HICKS): Yeah.

EAGGLESTON: I should know this but...

(LAUGHTER), that's to give this -- this concept a much farther reach than just the app challenge that we were running this time around.

The prizes for this that we did were $2,500 cash prize and then a trip for two to New Orleans to present their ideas in front of our house of delegates, which is kind of high profile for -- for the winners.

HICKS: We were really -- we were really afraid that that prize level was -- was terribly low and people wouldn't submit ideas for that -- that amount of prize. We were pleasantly surprised, though.

EAGGLESTON: Yeah, because when we started this, I mean, we're -- we're out there looking at other -- other Fortune 500 companies that did something similar, not necessarily around the apps, but they could have been giving away $20,000, $50,000, so little concerned about that one.

The submission process was also handled through the micro-site. We asked them several questions on how their app idea would actually work for their audience, for the AMA, and also to give us, if they had any visuals that they sketched out, to provide those for us to help tell the story.

P.R. was pretty good with this. We -- numerous stories picked this up because it was around apps, not necessarily because it was around the AMA or that the -- that what we were doing was -- was that interesting, but they really caught on to the medical app concept.

Go ahead.

Again, we promoted on our website and we also had different types of ads that could be used internally and externally. And email campaigns, both individuals, out to our member and non-member audience along with getting communications in any of our internal channels that we had, whether they be weekly, monthly, quarterly newsletters.

And collateral for this, we had pre-promotion and during promotion brochures and tabletop signs that we sent out to numerous events. Very little in actual print media but, again, taking advantage of the resources that we already had here at the AMA from the different publications that we already work on.

I did a video that we put up on YouTube that also helped with our press release getting picked up, the different channels or the blogs or Facebook or have you could use the video also and social media, through blogs, Twitter, Facebook, all the traditional channels that we could get.

And at different events, we -- we -- during the time of voting, we had access to about 25 events that we were already going to so this was kind of like a tag-along.

The photo there is -- I actually had a plasma TV kind of wrapped to look like a smartphone that I played the video on during our annual meeting, which is one of the largest collections of members.

HICKS: So the result of that is that we had 358 mostly excellent ideas submitted. You know, as you can see from our art segments that, you know, we -- we had some assumptions when we went in.

We thought that, oh, it was all gonna be the students and the residents, the younger crowd. We were really surprised that over half of the submitters were -- were -- were older practicing physicians.

Our oldest submitter was 85 and actually submitted a pretty decent idea, you know. And from our members, you know, there was a good amount of our non-members, people who are, you know, don't normally engage with the AMA, actually submitted ideas.

Now, we actually had 10,000, you know, close to 10,000 site visits and 26 of that came from -- percent, came from sources other than the AMA generated sites and links, so from outside the AMA.

Oh, OK. So -- so the next part was the vetting -- vetting and winnowing phase so that during that period of time in July, we actually narrowed the -- the 358 down to the top 30.

And we brought in a physician panel of -- of experts, folks from Stanford, you know, other innovators in this -- in the area and we presented the top 30 ideas and had sort of a day-long event and had them actually select the top 10 ideas.

We -- we rated all of them using some criteria, which is also in our rules. Basically, these were the things, we wanted it to be useful and engaging to our target audience, had to fit with the AMA's mission. We wanted something new, innovative and unique.

Had to be suitable for the app format, you know, and it should represent, you know, your expertise, your skill set, your background, what you're bringing as an idea submitter.

And then we turned it over to the external voting phase.


EAGGLESTON: So we changed over the micro-site to a voting platform. You could get to the site, read the -- the submissions, or our phrased-down versions of the submissions, look at a screenshot.

And then to actually vote on the idea, it switched over to a members-only login screen.

And these are the mock-ups of the 10 app ideas that we had. We had an app called Rounder, What's Not Covered, Mobile Surgery Card, Duty Hours, Medical Calculator, The JAMA Clinical Challenge, Physician Advocacy, Disaster Preparedness, Medical Flashcards, and Residency Tracker.

HICKS: Yeah, and -- and the interesting thing about these is that all the 10 that we've put up here are -- are -- we're -- we're honestly just, you know, we'd be happy to build any one of these. These are all really excellent ideas that were submitted to us.

And the mock-ups that you see there is we just had a professional artist do it so that there wasn't a bias in the voting as to, you know, someone who had -- didn't -- maybe didn't have the great, same artistic skills. We wanted
everybody to have an equal footing.

Additionally, when we put the ideas up for the members to vote on, we -- we randomly selected the -- presented the order every time you refreshed the screen, every time you came into the screen so it wasn't just the ones at top always getting, you know, clicked on.

It was, you know, you really had to choose the one that you -- you wanted to win when you were voting because you had to look through the list to find them.

EAGGLESTON: You know, the other thing to note here is that the -- the first row there of app ideas were from what we -- for the sake of this challenge, what we called practicing physicians.

The second row of app ideas were physicians-in-training, you know, from the student and resident population. And we were very pleasantly pleased to see the quality of the app ideas that were coming from the students and residents, that the apps were not just useable for students and residents. They were useable across physicians.

HICKS: Yeah.

EAGGLESTON: Again, here's just another little drilled-down screenshot that I think you guys will have access to the actual website, where you can read through the ideas, the top 10 anyways.

HICKS: So we had almost 2,000 members vote. Our voting was open for six weeks. It was a pretty, you know, we had a big rush at the beginning. We had a big rush at the end and then because of some of the marketing efforts, we had a few spikes in between.

We -- we allowed one vote per member, one vote in each of the categories. You know, we've -- we -- we did restrict the actual voting to our member population, which in some ways was a drawback because it was password protected and not all of our membership has the single sign-on to our website.

But we still had, you know, close to 12,000 site visits so we were really pleased with that.

EAGGLESTON: Right. So some of our takeaways with this, obviously, we've only shown you all the good stuff. There were numerous legal reviews. This type of thing was absolutely brand new for the AMA, very risk-avert organization.

When you're talking about actual prizes and we couldn't use the word, contest, and, you know, so again very -- we were very conservative and we worked through a lot of hurdles on that. So that was one thing that I would have been a little bit more prepared for on.

And then the -- the one good thing, which we -- we, our gut told us this, but it was nice to see that -- that this happened is that the prize money was not what drove them. They -- they did not -- no -- no one that won or submitted the idea, at least from what we can tell, actually needs the $2,500 or needs to have a trip paid for them to New Orleans.

This was really about recognition. This was about them sharing an idea, a concept that they -- that they felt that there was a need. Everybody, you know, there's a little inventor in all of us at some point. So this just gave them a fun way to share that.

And again, the marketing budget for this was miniscule. It was nothing compared to some of our competitors or the consultants that we had talked with on what to expect. We really tried to go after every channel that we could.

And we got a lot of earned media and a lot of help through that, which was really great, that was outside of things that we had to pay for.

And again, like Bob mentioned, the diversity of the submitters was unforeseen from states and specialties and the different lifecycles, submitters that probably hate -- I shouldn't say that, but submitters that probably don't think -- don't put the AMA in a positive light.

And definitely some areas of the country that might not always believe in what we're doing, but we were still getting submitters from those areas or from those specialties or those groups so that, for me, that was really the ultimate goal.

And unforeseen channels of collaboration and connecting with an institution such as doing this and working now with Stanford on a couple of things and again, it was not only an innovation and engagement challenge from the AMA to physicians, but it's now become from the AMA to other organizations, which is very exciting.

Because we're -- we're really looking to expand and have this only be the beginning of something greater.

HICKS: And for 2012, what are we doing? Well, as Eric was talking about earlier, you know, this -- although it may have looked high-tech, this is really decidedly a low-tech sort of challenge.

It -- it was just a -- a -- a web-based version of the recipe contest or the jingle contest that my -- my parents used to participate in. And, you know, we -- we -- we specifically kept it that way.

But it -- it really has opened up a channel for us. That's why we, you know, we -- we claimed the domain name, We're hoping to sort of create an open innovation portal like Procter & Gamble and some of the other companies -- G.E. I believe is also doing.

And -- and it's the way to share data with our stakeholders. It's a way to share ideas with our stakeholders. And it's a way to engage them in ways that we don't traditionally do.

You know, we engage them along advocacy. We engage them along, you know, issues of health and whatnot. But from an innovation perspective, we really haven't done it in sort of the non-clinical, non-academic setting before.

And this has -- this has been a learning experience for us all. We're gonna take it nice and slow. We have other ideas for additional challenges in 2012. We are going to explore some of these idea platforms and prediction markets that Chris talked about in the coming year.

And we'll -- we'll see what happens. We're -- we're excited about the -- the direction that it went in.