Today’s post was co-written with Melissa Davies, Healthcare Research Director at Nielsen in the Online Division.

Key Points Summary (detailed post follows)

• Reportable adverse events are far less common than most people suspect. There are only approximately 166 reportable adverse events per day recorded across the entire pharma industry.
• Even for the biggest pharma brands, there are very few discussion happening online that include a mention about the brand. Only 36 per day for the biggest selling drug in the world, Lipitor.
• When you consider the number of discussions that might have a reportable adverse event, it would take a long time to find one for most brands. For example, you can expect one reportable adverse event every 2 weeks for Lipitor (for the site monitored in this study).
• For many pharma companies, it would be difficult, but not impossible to do this monitoring in house, as some automatic filtering could simplify things. Using call center employees, who are already trained in how to handle these discussions makes sense.
• There are three categories of discussions that need to be screened: those you could skim past relatively quickly, those that might give pause for thought but could probably be filed away or handled with pre-approved response language, and those that may require escalation depending on the company’s social media and customer response policies. The time required to screen all discussions depends on the mix of these categories.
• Companies need to determine what and why they are monitoring. This means determining if you’ll monitor only your sites or which 3rd party sites as well. Finding adverse events isn’t the only reason. Correcting misinformation, understanding patient needs, and engaging in dialogue (e.g., answering questions) are also reasons.
• If companies are only monitoring their properties, expect a very low number of discussions and even few discussions that might be considered reportable adverse events.

Detailed Post

In a first for Dose of Digital, today’s post was co-written with Melissa Davies, Healthcare Research Director at Nielsen in the Online Division. Not familiar with Melissa, you say? I bet you’re familiar with her work especially if you’re  a regular reader of this blog. Melissa was the lead author for the now famous “1 in 500″ white paper about the incidence of adverse events (AEs) in social media. This is the report that showed the Internet isn’t chock full of reportable adverse events just waiting for the first unsuspecting pharma company to happen by and be crushed by the deluge. Rather, Melissa and her team’s work showed that only 1 in 500 (0.2%) randomly selected discussions (blogs, comments, forum posts, etc.) contained all four criteria required by the FDA to be considered a reportable adverse event. If you want more detail on this then check out my post The Myth of Adverse Event Reporting and definitely get the original paper (PDF) from Nielsen now as well.

Those are some pretty strong credentials to help me out with a follow up to my recent post 166 Reportable Adverse Events Equals One Red Herring. To create that post, I asked Melissa to supply some more information about the original Nielsen study. The reason for this is because in discussions with many people from the pharma industry, I discovered that many knew the “1 in 500″ stat, but remained concerned about the volume of AEs out there. Their rationale was simple: 0.2% of, say, 50 billion is still a pretty big number. Without knowing the number of total discussions, the 0.2% number doesn’t mean much. That meant figuring out how many new discussions are generated each day (and it’s not 50 billion).

The answer came from Melissa, who dug a bit into Nielsen’s database to answer a seemingly simple question: how many new pieces of healthcare-related content are generated each day online? Nielsen monitors 1,350 sites that it considers to be healthcare-specific (and millions of non-healthcare sites too). Looking at these 1,350 sites over time showed that, on average, there are more than 83,000 new pieces of content generated each day on these healthcare-specific sites. So, at least we know it’s not 50 billion.

That was the inspiration for my post. 0.2% X 83,000 = 166. There are 166 reportable adverse events generated each day for the entire pharma industry to handle. 166 isn’t a lot to me when you divide the work to manage these events across all the companies out there. Yes, larger companies are likely to have more and controversial products might also have more, but 166 for everyone to share is a pretty manageable number.

Almost immediately after this post went live, I was contacted by several pharma companies looking for some more information. Specifically, many weren’t convinced that it was quite this simple. That is, it may be one thing for Nielsen to scan through 500 messages and come up with 0.2% and to show the total volume, but it’s quite another for a pharma company to screen the more than 83,000 new pieces of content by hand themselves each day. How could one company possibly screen every single one of these pieces of content each day to find the few bits that refer to their products? What time commitment would be required to find these needles in all these virtual haystacks? When it comes to a particular brand, should they expect to find a few adverse events? Dozens? Hundreds?

I saw another objection to social media igniting before my eyes, so I decided to stomp out the flames before they got out of control.

I went back to Melissa and asked if they’d be interested in doing some more detailed analysis to show that the volume of drug mentions for any brand is quite manageable. They agreed and the result is this post. As I mentioned already, quite a few companies asked me after my “166″ post for the volume of discussions for their products. However, as this is how the folks at Nielsen make a living, we weren’t able to do with this. Instead, we decided to do a random selection of three companies from the top ten US pharma companies. The winners of this little lottery were Lilly, GSK, and Pfizer.

For each company, Nielsen looked at the top-selling products for this analysis. From their dataset of healthcare-specific websites, Nielsen BuzzMetrics collected, on average, more than 83,000 new discussions per day for the first half of 2009. Within this, there are a number of discussions about theses top-selling products. So, without further fanfare, here’s a look at the average number of discussions per day for the top five brands from each of the selected companies:

[Click on the image for a larger view]

Across these 15 brands, there are an average of 45.4 online discussions per day incorporating a brand mention. The volume of discussion can vary widely by brand. One interesting revelation: top sellers are not necessarily the most-buzzed brands. Lipitor and Advair, for example, are the two best-selling drugs among the 15 brands in the US (in dollar sales) and yet fall squarely in the middle of the pack for mentions. The top-mentioned product was Lyrica, which came in at number 8 of 15 in product sales.

Of course, there are many factors that can affect conversation volume, including disease state. It is interesting to see that some of the products you might expect to have a lot of volume based on their “controversial” nature don’t rise to the top. Cialis and Celebrex immediately come to mind. For the former, you might expect more off-color discussions cropping up, which would drive volume, but keep in mind, Nielsen only reviewed healthcare-specific sites for this analysis (so, any mention of Cialis on, say, PerezHilton.com, isn’t going to show up). In addition, mentions that included obvious spam terms such as “buy online” were excluded. For Celebrex, you might expect more volume based on the product’s past controversies. This doesn’t appear to be the case. However, volume can also change suddenly – when there is news about a brand (bad or good), a new market entrant, a public event related to the condition (Breast Cancer Awareness Month), etc.

So, that’s the raw data, the question now becomes: If a pharmaceutical brand wanted to monitor all of this discussion for things like adverse events, claim expansion, misinformation – or even just to understand what consumers are saying about the brand – can it be done in a practical manner?

To start, some filtering can be used to automate part of the process. For example, messages can be filtered for mentions of brand keywords. That’s what was done for this analysis. Nielsen filtered the more than 83,000 messages and pulled out only those messages that contained one of the 15 products selected for this analysis. It’s a very simple filter that every basic screening and monitoring tool can handle. The rationale for filtering out discussions that don’t contain a mention of a brand is that with this, you can’t have a reportable adverse event.

The next step is then sifting through all the mentions of your brand. The chart below shows how many discussions per day, on average, each brand would have to manage. For Cialis, about 17 discussions would have to be screened each day. For Lyrica, on the other hand, 132 discussions per day would have to be screened.

When you apply the “1 in 500″ statistic to these numbers, you get a better sense of how often a reportable adverse event is likely to show up.

[Click the image to enlarge]

Translation: It takes a long time before a discussion with a reportable adverse event pops up. For example, it would take almost a month’s (29.6 days) worth of discussions to find one Geodon reportable adverse event. At the same time, you might expect to find one for Cymbalta every 5 days or so. Two things come to mind: first, that’s a lot of discussions to review without finding anything. Second, rather than being a deluge of  reportable adverse events, most brands aren’t even talked about that much making it much less likely that there are reportable AEs out there that you’re missing.

Since you might have to go through quite a few discussions to find one that requires your attention, let’s see at what the screening process might look like. First, the time required will vary greatly. Some discussions can almost instantly be determined not to have a reportable AE, while others are going to require some more time and attention.

Taking a look at some of the discussions seen for Lyrica (the most mentioned of the brands reviewed), you can basically group the discussions into one of three categories: those you could skim past relatively quickly (as they lack even the most basic information), those that might give pause for thought but could probably be filed away or handled with pre-approved response language (if you were doing actual outreach and engagement), and those that may require escalation depending on the company’s social media and customer response policies (these are the Discussions with potential reportable adverse events).

Here’s one of each from Lyrica [note: quotes are unedited]:

Skim past quickly: “i am doing much better thanks hun…i have neuropathy and use lyrica as well as ativan it does help me / lyrica helps with fibro….my very best to you and hubby…happy holidays sweetie” – from healingwell.com

Requires some thinking: “Aren’t they basically the same drug? Lyrica is just FAR more powerful than Neurontin? My Neuro explained Lyrica is 8x more powerful than Neurontin and if he up’s the Neruontin and makes an equiv. does it will be about the same response?” – from neurotalk.psychcentral.com

Might require escalation: “I am on my second try with Lyrica. I am on 75mg twice a day. I had dizzyness and blurred vision both times. The dizzyness has subsided, but the blurry vision is still there. I am on my 3rd week and so far I haven’t noticed any pain relief. Using it in combination with Nortriptylene and Lorcet for nerve pain in my ribs.” – from healingwell.com

So, the screening and coding for the 132 Lyrica discussions might only take a few minutes or could take an entire day and several people’s efforts if they are sufficiently complex. It all depends on how many of each category you have.

Beyond simply filtering for brand names, it’s possible to automate the process by looking for keywords related to known side effects, and/or keywords related to negative perception. However, this step brings up two key challenges. First is that any unknown side effects (which are the ones a brand might be most interested in discovering through this process) are the true “needles in the haystack” since they will likely not be covered by established keywords. That is, you’re not likely to find the unexpected and serious events that can really impact public health if only look for known side effects. Second is the challenge of using natural-language processing to accurately detect sentiment around healthcare messages. The nuances and unique considerations within healthcare discussion make it very tough to train a computer to digest what patients and caregivers care most about. Many monitoring companies, including Nielsen BuzzMetrics, use keywords to identify messages about a brand, and then use manual analysis to read and code messages for sentiment and topics of discussion. Manual analysis means that someone has to go through all the messages by hand at some point and determine what’s important and what’s not.

A pharmaceutical company could consider doing analysis process internally. Social media messages could be automatically screened for mentions of particular brand names, and then sent to a team within the company for review and follow-up action, if warranted. Within BuzzMetrics, they typically find that an analyst can read and code about 100 messages per day. That doesn’t include any internal routing or follow-up communications with original posters that a pharma company might want its employees to do, which would take additional time. An ideal group to handle this at pharma companies are those people already staffing your call center and who deal with adverse event reports received via phone (and other product inquiries). While they aren’t on a call or otherwise have a lull in the action, each person can review a handful of messages and determine if any need action. If they do find one, then the information is already in the hands of the right people at the company. Call center reps are trained on which issues they can handle directly and which need to be escalated. They understand the chain of command. They have established scripts to use over the phone, and some of this language (or key themes from it) can translate to the online world. And, of key importance, they know how to interact one-to-one with customers – which is really what social media is all about.

Let’s also keep one thing in mind. This volume of mentions is basically for all the social media discussions on all the health-related sites on the Internet (English-speaking only). These sites range from massive (like WebMD) to personal blogs with small followings. So, if you choose to monitor everything out there, this data shows what you can expect. However, under the current regulations, you are not required nor obligated to monitor third-party sites unless you are somehow connected with the site (as a sponsor, etc.). This means that you are only required to manage the sites that you maintain, own, or otherwise control in some way. This includes assets like your brand websites, any blogs, YouTube channels, Twitter (if someone DMs or replies directly to you), Facebook pages, and unbranded disease information and community sites. If you already have any of these, you know that you aren’t going to get a huge volume of discussions whether they be blog comments or YouTube comments. Most of the pharma social media programs that I’ve seen have received only a handful of comments over their entire lifetime. Even the most ambitious and well-known properties such as J&J’s BTW blog (the best healthcare industry blog IMO) don’t get very many comments. Their last 10 posts have 13 comments combined (and 6 of these came from Marc Monseau‘s post “What’s the ROI?”). That’s 13 comments since October 27…not a lot to monitor. Of course, J&J might have gotten more comments than this and a few were removed in moderation, but based on what I know of this blog very, very few comments are not published.

The next question is: “why?” Why are you screening and scanning the entire Internet looking for adverse events? The answer might be simple. Perhaps you want to know what people are saying about your brand to help direct future communications. Maybe you’re actually going to talk back and not just listen in. You might also want to listen everywhere because of a genuine desire (whether legally required or not) to know everything you can about the safety of your product. You might be looking just to check for unexpected adverse events. Isn’t it better to find out early about a serious adverse event that keeps occurring and, yet, wasn’t seen in clinical trials? The longer you wait, the more lawsuits get lined up. Of course, more important than the legal issues, the longer you wait or the longer it takes to discover a new, serious adverse event, the more people who could be harmed or killed by the product.

Perhaps the other answer to “why?” might simply be to provide better service to customers by better understanding them. Not just to know about harmful effects, but to know about questions, misinformation, patient concerns about the medication, or about disease treatment in general. You can see from this and many other analyses that there isn’t a high volume of AEs in online conversation, so if we move past that, maybe better serving patients should be the ultimate goal of monitoring online discussions.

We’ve found that most companies actually do want to monitor everything that’s out there (before you do, read my post Why Pharma Should Forget About Social Media Monitoring), but they either lack a purpose behind their monitoring or they’re concerned about what they’ll find. The latter concern includes worrying about how to manage the volume of messages that are out there. However, this analysis shows that the volume isn’t unmanageable even for the biggest pharma brands and perhaps eliminates one more barrier to pharma and healthcare companies first observing and then participating in the discussions happening all around them.