The Great Social Experiment – Roundtable Question 2: Judging Outcomes

REPOST from original series started on back in September of 2007

Back in September I began a series on experimenting with social networks to view their effectiveness in helping organizations expand their reach, and be included in conversations about their industry wherever those conversations occur.

Now it is time to look at the results. The other day I introduced the panel (Aaron Brazell, Chris Brogan, Doug Haslam, Cathryn Hrudicka and Brian Solis) and the questions on the table.

Today the second question: How should we be judging the outcomes from social networking activities? Are there tools you recommend?

Chris Brogan
If you’re asking how to track who’s saying what about you, I highly recommend Radian6. Great new tool that does a lot of what I used to do with Technorati and Google Blogsearch.

Doug Haslam
As a PR practitioner, I hear “measurement�? as that big scary word that makes us hide under our desks, or behind large piles of cow manure (Brian may disagree, but he’s smarter than I am). Any measurement needs to be taken back to basics: what are you measuring? Will traffic generation suffice if you are really looking for leads? Will leads suffice if you are looking for revenues or downloads? How can you measure conversations and relationships?

That’s right, your question begets more questions, and is complicated by the way you spread your campaign over several social networking platforms. You have me thinking about how I can do that better for my own campaigns. So, some thoughts about measurement:

• You were right to demand metrics from the social networks. I doubt you will get them from the providers, unfortunately, but we can hope. The ones that do will forge some very interesting business relationships.

• Since you are experimenting with different social networks, you will probably have to come up with your own metrics for each one. Perhaps you can rate effectiveness per “friend�? in each network. How many inquiries/leads/downloads/sales do you get per friend? Did you have better luck on Facebook or MySpace with those metrics?

• Against my better judgment, or perhaps because of it, I love to go with my gut. If the numbers don’t smell right—if you think something is worth doing despite your metrics—you may be right. Just think about how you are going to sell that gut feeling to your client (or boss).

• More than three bullets in any report of recommendations better be worth the extra reading

As for tools: again, it depends what you are looking for. If you are monitoring online reactions and sentiment, several tools with different features, like Custom Scoop, Umbria, or Buzzlogic, depending on your budget (and other services can get really pricey). Caveat: many of these are media monitoring that extend to –or focus on—blogs (I told you I was PR guy), but the evolution to social networks may already be in place. Perhaps a simple blog analytics tool will suffice? The field is still really an early-growth forest.

Cathryn Hrudicka

There are qualitative and quantitative metrics for results, and since social networking is still relatively new, it is particularly more difficult to get accurate quantitative metrics. The various social media sites do not all provide demographics or accurate numbers, and the numbers change frequently. Many of these sites are likely to come and go within the next few years, as new types of social media are being developed all the time. I’m a member of the Social Media Club, and several members of it and other progressive public relations organizations are trying to develop (and continually update) metrics for social media.

There have recently been some interesting articles on the topic: Kenton
Robert Scoble Gilliatt Brogan – Fast Company article by Robert Scoble Huyse Johnston

On a qualitative level, I think you can judge outcomes by noting how other social network members are perceiving your brand and company, if it is positive—and if negative, how much you are doing to change their perception by authentically conversing with them and adapting to a more positive direction.

If other key influencers in your social networks are blogging, podcasting or video blogging about your company, products and services with enthusiasm, then your social networking investment is worth it.

What is especially valuable is the viral effect, which you can measure—here are some questions to ask yourself about your social media ROI:

• Is the media coming to you as an expert, rather than your having to constantly do PR outreach? In other words, are you being asked for interviews by all types of media (print, broadcast, trades, wire services, in addition to online), as a result of their finding you on social networks or seeing and hearing about you on blogs, podcasts, and other social media?

• Are you being asked to join expert industry roundtables, guest blog by popular bloggers, or speak at conferences and other events as a result of your presence on social networks?

• Are you seeing a significant increase in number of new potential clients or customers who at least contact your company for information?

• Do you have a significant increase web site and blog traffic, podcast downloads, and RSS subscriptions following your social media campaign?

• When you do regular Google and other searches, do find your SEO (search engine optimization) has significantly increased? Some social media sites definitely boost your SEO and the number of back links to your web site and blog. Are you using Google Analytics and other tools to measure your site and blog traffic?

• Have your Digg, Technorati, Alexa, or StumbleUpon rankings increased significantly?

• Are you or your company being increasingly referenced as a “thought leader�? or “industry leader�? or “industry expert�? since you began your social media campaign?

These are just a few areas where you should be able to measure metrics. The viral aspect of having your quotes, articles or blog posts about you jump from one type of media to another, and move from local to national to international coverage, is a key marker to watch.

Brian Solis
As I mentioned in my last post, in order for businesses to understand Social Media, it requires a realistic strategy, goals, metrics, and a tactical plan. It all starts with answering two questions, why do we want to participate and what do we want to get out of it?

This is usually where traditional media and new media split. Traditional marketing is rooted in tangible results such as press coverage, traffic, revenue and those things should still matter in conversational marketing. This can’t be solely driven by experimentation because time is money!

Metrics must be determined before engaging and then actively monitored and analyzed to modify future interactions and targets to stay on track.

However, there is an investment component in all of this that’s somewhat comparable to the model of having a great customer service infrastructure. It’s actually a significant cost center to all businesses that care about having a global community of happy, satisfied, and enthusiastic customers. The difference is that with Social Media, an outbound element is added to the traditional inbound model.

No it’s not a copout for aligning metrics to engagement. It is, though, a different way to look at things. And, it brings the discussion outside of just PR and marketing. Meaningful engagement can also be led by product marketing and customer service.

Last year I discussed the concept of scientifically analyzing the Return on Participation (ROP) or Return on Engagement (ROE), which called for feedback to help determine what it is that matters to businesses in order to justify the expense of running social media campaigns.

The easy answer is as true today as it was then. It’s different for each company.

It’s also different specific to the campaign you’re running.

I think it creates new line items that are separate from traditional marketing and should receive funding and resources from both marketing and customer service budgets.

In general, some of the most effective ways that we’ve measured success to date include community feedback, trackable conversations, traffic through landing pages, referrals, and also registrations specific to campaigns. Really, it’s similar in concept to Web marketing. However, measuring conversations and tracking associated activity is an interesting and creative art.

The tools that we use to track activity and conversations include Technorati, Twitter, blogpulse, Google Blog Search, Compete, Google Analytics, Alexa, specific social network searches, among others.

What’s the value of a conversation?

What’s the value of transforming an unhappy customer into an enthusiast?

Many companies can learn from big businesses who are shifting their reward-based strategies from who can take the most calls in an hour to who can create the most enthusiasts.

Like I’ve said, customer service is the new, new marketing and there’s a lot to learn from separating (or expanding) engagement from PR to those immersed in the product, its value, benefits, strengths, and weaknesses.

Measurement is a combination of customer service, brand resonance, brand loyalty and bottom line business traction and lead generation.

And like I’ve always said, businesses will earn the respect, and ultimately the business, of the customers they deserve.

Aaron Brazell
Determine your desired goal before you even begin. If you are looking for that Latin American business partner, then your goal in using the social sphere will always be finding and hiring that partner. Maybe it’s not even hiring. Maybe it’s building the relationship necessary with him.

If you’re looking for traffic to a story, then getting on the front page of Digg is probably your means to success. Failing to do so may not mean you failed. You just might try StumbleUpon instead. If you fail to get the eyeballs, do you call it a total failure? I don’t know.

Another great round of answers. The main points I take away from all this advice is:

  1. Understand your goals to begin with.
  2. Know the tools you are using and their target demographics (which is a great segue to the next article) and be selective.
  3. Use whatever analytical tools those services have, but don’t be afraid to use your own self-defined measurements, or as Doug said, use your gut.
  4. Try, test, repeat. If one tool doesn’t work try the next.

As many have said I think I spread myself too thin. Being able to adequately judge outcomes based on efforts takes time and focus. Without that high level of focus you both decrease your chances of success, and decrease your ability to measure.

The conversation on Monday will turn to understanding expectations and finding the right tools for the job.

Social Networking round-table articles:

  1. Did I devote enough time to the exercise? More to the point, where should social networking be placed in terms of marketing priorities? Does this change for companies that are more or less mature?
  2. How should we be judging the outcomes from social networking activities? Are there tools you recommend?
  3. Is social networking the right tool for the tests I created? Are my expectations of social networking in line with what it can do? What is social networking best geared to address?
  4. How much of getting social networking to work is the contacts you bring with you? How aggressive should one be in reaching out to new contacts? Should I, for instance, try to befriend Chris Anderson even after I have composed a note meant to appeal to what he is interested in?
  5. How can I build better mechanisms into the framework to increase feedback?

Previous blog articles in this series:

Can you digg it? I knew that you could – The Great Social Experiment

The Great Social Experiment – #1 Getting Started

The Great Social Experiment #2 – Will You Be My Friend

The Great Social Experiment – #3 owwww, i’m u’r biggest fan

The Great Social Experiment – How has it gone?

Oh and don’t forget to use the social networking tools below to share and enjoy – part of the experiment right?



2 Responses

  1. Interesting perspectives here, but I think you need to point out that the tools that you mention only monitor, they don’t measure. You also totally left out the web analytics picture. Measurement looks not just at the content of what’s being said about you, but also correlates its with your stated goals and objectives. Integrating web analytics with monitoring gives a much more complete picture of what is really happening. Here’s my take on the topic:

  2. Katie,

    How do you feel that the tools we mentioned only monitor and don’t measure. Just looking at two, Umbria and Buzzlogic, they both have measurement tools, analysis and metrics.

    As for web analytics, we did kind of gloss over that one. Several times the panel mentioned traffic. I don’t want to speak for everyone, but on my part there was an underlying assumption that if you are measuring your SN activity you are probably already using web analytics. But I think you are right in that it can be much more than just understanding *how* many people are coming to you site because of any given SN activity, but there is knowledge to be had in their specific actions coming from your particular SN interaction with them.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: