PR Conversations welcomes a guest post by Bill Sledzik, associate professor, School of Journalism & Mass Communication, Kent State University, Ohio, U.S.A. Collectively, we quite like his regular blog posts (i.e., thought process, rigour and sense of humour), and felt that our international readership would enjoy a taste of some Tough Sledding, too.
When I started blogging back in 2006, I promised to continue only if I saw return on investment (ROI). Today, 150 posts and 745 comments later, I’m still a blogger. Tough Sledding has delivered ROI, both for me and for the PR program at Kent State. I’m sure of it. Kind of. No matter how hard I try, I can’t find a meaningful measure of my blog’s return on investment.
I could cite anecdotal evidence. For example, my posts have triggered six interviews with mainstream media (two of them U.S. national). I’ve also been quoted and linked to by nearly 100 other blogs. Overall, my body of work gives me a decent profile in the PR blogosphere—nowhere near the “A” list, but I have a small following and great Google juice.
But what does it really mean—any of it? And what has really changed as a result of my social media investment? I hope every organization is asking these questions before they spend loads of time and money on social media programs. There’s a lot to be gained in Web 2.0 experimentation, and I encourage it. But in the end, we’re all held accountable for how we spend the organization’s time and money. So focus on ROI.
The question of how we measure social media isn’t one I can answer. But I hope to suggest a simple direction and pose some questions for discussion.
What really matters?
Case by case we do a decent job tracking outcomes of social media programs. Go back to the fascinating story of the English tailor that Shel Israel wrote about. That guy sold some suits, reaping real ROI from his blog. Fast-forward to the more recent case studies that Geoff Livingston urged me to read. Good examples all, but only examples. And because results are self-reported by the consultants who implement the campaigns, they’re a little shy on objectivity. The consultants also don’t post stories about campaigns that fail. It’s bad for business.
So where do we go after the case studies? I’m not a measurement expert, nor do I understand the science of algorithms. But like most PR practitioners and educators, I know the value of return on investment (ROI). Management has been calling for bottom-line measures from PR for the past 25 years. I’m old enough to recall the transition from simple measures of column inches and gross impressions to the more tangible focus on behaviors. How effectively does our work generate leads, drive sales, reduce costs? All are bottom-line outcomes, and often difficult to tie back to PR activity—online or off.
What about relationships?
How do we measure what we really do—the relationships. If you accept the two-way symmetrical model—and I do—you embrace public relation’s mission as the building and maintaining of relationships that help our clients succeed. We help create environments that foster success. We help connect organizations and stakeholders to assure prosperity. Measure that!
As Brian Oberkirch declared earlier this year, it’s a bit like trying to “nail down a shadow.” But nail it down we must, or our social media initiatives won’t be taken seriously by the people who pay for them.
The PR measures of yesteryear won’t cut it—circulation, impressions, households. Nor will the modern gauges of page views, click-throughs, etc.; they are already considered passé by the experts. Leads are great, because we can track them directly to sales in some cases. But not all PR campaigns are designed to generate sales.
Oberkirch lists 22 measures we can consider for our social media, but only three will turn the heads of CEOs:
• sales
• cross sales
• reduced service costs
These are all bottom line outcomes. The rest—page views, feed subscriptions, comments, etc.,—are important for our own tracking operations, but of little value in the boardroom.
The more I read blogs and follow the use of social networks by business, the more I believe measuring social media isn’t much different than measuring other PR activities. It’s all about the behaviors, something Pat Jackson preached until the day he died. (This was along with Jackson’s belief that relationships are at the core of effective public relations practice.)
The steps leading up to behavior may be easier to observe online, thanks to the digital fingerprints we leave. But in the end, a PR professional still wants to know if the stakeholders did what we hoped they would do: buy, sell, vote, etc.
Where to go from here?
Because those behaviors so often result from relationships PR folks help to build, we’re back full-circle. But can we ever really measure the “relationship?” I won’t go all “academic” on you here, but every PR professional should read the seminal paper on this topic from Grunig and Hon. And while you’re on the Institute for Public Relations’ website, surf around to find other papers that came later, such as those papers testing the premise of relationship measurement. It may spark and idea or two for measures we can build into our own social media initiatives.
It’ll take someone far wiser than me to operationalize concepts like “control mutuality” or “exchange relationship,” but it’s time to bring these ideas out of the lab. The good news is that all six components of relationships would be evident in just about every form of online dialogue, from e-mail to Facebook. We just have to capture the data. No easy feat, but do-able nonetheless.
One thing is certain: It’s time to take the task of social media measurement beyond the anecdotal. I’ve seen so many case studies I feel like the Web 2.0 world has gone into reruns.
Meanwhile, there’s no shortage of companies to help you measure social media activities. Just don’t look for any of them to measure behaviors or relationships. I am impressed with Buzz Logic, a company that touts an algorithm including more than a dozen factors to gauge the influence of people discussing a given topic in social media. According to its website, the system measures factor in, “relevance and overall popularity of an individual post, along with the relevance and popularity of all in-linking posts.…”
I still don’t hear the word “behavior.” But this sounds like a good diagnostic tool that can help reshape our strategy as it unfolds, and it’s relatively inexpensive. I’d love to hear what others are using to track and analyze the online conversation.
Answers? Who has answers?
Like I said, I didn’t promise answers to the measurement puzzle. If I had them, I’d quit blogging immediately and have my checks sent to the beach house in St. Lucia. What I’m hoping for is discussion on how we can connect, as systematically and directly as possible, social media activity to the behaviors of our stakeholders. If you have ideas on relationship measures, let’s hear that, too.
We all know the business world is built on measurable and tangible results. I worry that too many social media enthusiasts are overlooking this point. For most top executives, if you can’t weigh it or you can’t measure it, it doesn’t exist. I guess that’s why so few philosophers—and even fewer bloggers—are sitting in the CEO’s chair.
* * *
Related links:
About Bill Sledzik
Tough Sledding (Bill’s blog)
PR Heavyweights in 13-Round Slugfest (Bill’s guest “referee” gig on Strumpette)
Kent State University’s Public Relations Program
Media Mindsets research group at Kent State (research across all disciplines, not just PR)
No question about that, Heather. Call me greedy, but I’d like to find a way to measure the impact of the adaptation – of the win-win. Problem is, the benefits of adaptation tend to be long term (with the possible exception of crisis response). But as Kami pointed out as the thread started, more and more CEOs appear to be embracing the value of relationships, which is interesting, since the job of CEO is so seldom “long term” these days.
It gets tricky when you mix in the network of relationships we strive to maintain. In my classes, I often use the very simple example of a factory closing to illustrate. Investors may celebrate a closing, since it drives down costs. Customers may celebrate the lower prices that result. Employees and communities affected take a very different view, and for good reason.
In the end, we PR types must find a way to communicate the tough decisions that accompany sustainability. But somewhere along the line, relationships suffer.
Bill – helping management recognise the need to accommodate and adapt has to be part of a realistic approach to helping organsations be successful. That’s not to deny the importance of evaluation, but our services will gain more respect if our counsel shows the real benefits of win-win rather than win at all costs.
I’m back online after a relaxing weekend of honky-tonkin’ in Nashville. So let me catch up on the discussion before I catch a flight to St. Louis today. Sigh.
Andrew: I didn’t mention the “conversation index” because I can’t translate it into real value (with apologies to Stowe Boyd). It does tell me conversations are happening, but I’m not sure that translates to any meaningful measure of attitude or behavior change. Mix in some content analysis and the index has a bit more meaning. That said, I’m pleased with my own index at ToughSledding, about 5 comments to every post (if you count my own).
João: Great feedback. I have a tendency to focus on those measures that play well in the C suites. But you are absolutely correct. Our measures must be matched to the task. Most top executives aren’t concerned with our traffic measures, yet we know that the longer someone spends on a site the more likely that person has become educated on our message. And it is that education that translates to the supporting behaviors we later seek.
Heather & João: There is always a danger when your focus is on “our” publics or stakeholders. We need to push back from the “table” and give our management’s a balanced focus and to insist that not all measures be focused only on “bottom-line” behaviors that benefit the organization. But unless we can show how our activities make the organization more successful, it’s difficult to sell our services to top managers. Getting beyond bi-directional symmetry will be a lot easier if have measures that show organizational impact. That’s self-centered, to be sure, but I think a more realistic approach.
João – good point on publics. I always differentiate between stakeholders and publics on the basis that the former are seen in relation to an organisation, where publics form themselves around an issue or matter of interest to them. This helps emphasise that they don’t “belong” to the organisation.
I agree on the need to go further than bi-directional symmetry – which of course links into networks and more complicated relationships. Thinking back to the original social media topic of this post – that of course, is the relevant aspect that we need to understand and evaluate.
Hi Heather,
Thaks for the link to that discussion. I second Tom’s view when he says that he doesn’t agree with the attempt made by Jim Nail to substitute Output, Outtake and Outcome by Media Influence, Audience Influence and Business Influence.
I couldn’t agree more with you on the idea that objectives of our action shouldn’t be organisation-centered (in fact, I’m a great critic of the idea behind the expression “my organisation’s publics” because often we tend to believe that the publics belog to organisations). In fact, I’ve recently wrote about the fact that bi-directional symmetry is no longer enough as decisions satisfying both a focal organisation and a specific public might generate damages to other publics or to society itself.
João – See Tom Watson’s post http://dummyspit.wordpress.com/2007/11/01/output-out-take-and-outcome-on-the-way-out/ which links to Jim Nail, who is arguing against Output , Out-take and Outcome as general and non-specific (at http://blog.cymfony.com/2007/10/outputs-and-out.html)
I think we also need to take care in these methods and reflections on evaluation that success isn’t just considered from the perspective of the organisation.
If we are genuinely interested in two-way communications, one issue with a focus on achieving defined corporate objectives is that these may need to adapt as a result of the process of discussion and relationship building.
Decision makers will need to be counselled that an adaptive strategy may mean objectives accommodating the needs of publics, not just an asymmetric approach in satisfying its own needs.
Great post from Bill, and thank you Judy for bringing another extraordinary interlocutor to this “floor”.
I just downloaded Katie Payne’s book, thanks for mentioning it Karen.
From a conceptual standpoint, I think we should use more often the different professional roles (using Benita’s proposition, I like to speak in terms of Strategist, Manager, and Technician) to comment on PR concepts and issues, like that of Measurement, at different levels.
It’s clear that the bottom-line impact of investment is a concern of the C suite executives, and that the PR Strategist should be able to walk that talk. But it is also true that in order to do that, he must be helped by a lot of other measures made at different levels by Managers and Technicians. For example, the sales, cross sales and reduced services cost measures (assuming they are relevant indicators for the Strategist) must be broken down to small measures in order to be fully measured (e.g. You know each sale comes from a prospect and that you need website visitors to generate prospects, and so on…) – You see, I’m speaking here about difference between outputs, outcomes, outtakes and outgrowths.
On the other hand, it seems to me that we are mostly discussing the value of information. (and I quite often find myself talking to people who to forget that processing information is one of the most important tasks of PR within organisations). Speaking from a semantical point of view and not from the point of view of the Mathematical Theory of Information, the general value of information on measurement depends on how you’re able to relate it to your objectives. Saying that your website had an increase in visitors, time-spent on site, average depth of the visit and page views, etc. might be boring and uninteresting for a top level executive. But if you are running a campaign and you proposed to spend money to achieve an increase of X % in the period Y in those indicators, then it becomes a very important information. So probably the best way to make your measures relevant to decision-makers is by relating it to pre-agreed objectives.
Good discussion here, but I didn’t see anything about the “conversation index” as a possible measurement tool. This is another way of measuring the value of blog posts in terms of comments it elicits. See http://www.stoweboyd.com/message/2006/02/the_social_scal.html for further discussion about the CI.
Andrew Careaga,
who wishes the CI were higher on his blog
Toni, if any of your four clients are making use of social media *and* including relationship management/measurement into the mix, this would be a good place to detail the case study or studies.
If you are referring to more traditional PR practices, likely that is more deserving of an original post by you.
Effective today (Friday), Bill is on an extended (and mainly offline) weekend of travel and fun. I doubt he will be checking in again until Tuesday. In the meantime, I’d encourage you to continue the discussion. Bill would particularly appreciate hearing success stories from non-(North) American readers of PRC.
I tend to agree with Kami that there is out there a growing number of CEOs'(from private, public and social sectors) who are investing significant efforts in stakeholder relationship management systems and are therefore keenly interested, first of all, in evaluating the dynamics of the state of their relationships with key stakeholders.
I, just to cite my ongoing experience, am presently engaged professionally in five different and complex programs of this nature for as many organizations (one social, one public, two private) and they are all embedded with periodic analysis done by Doxa (an Italian research company) on the dynamics of relationships with key stakeholders (suppliers, employees, media, retail and institutional shareholders, business leaders, public policy makers) following the trust, satisfaction, committment and power balance indicators specifically on the individual relationship rather than the organization.
In two of these cases year end bonus of their board directors are also connected to research outcomes.
I am sure there are many more of these cases in my and other countries.
Personally I believe that I could describe my cases in greater details if readers are interested and willing to describe theirs.
It would be good for our profession. Opinions?
Kami: I’m only to glad to have turned you on to Pat Jackson. Sad thing is, there isn’t much in the digital archive — at least not that’s available to public access. Too bad, as Pat’s reign as editor of prreporter turned him into a true guru for the behavioral model of PR practice and the symmetrical paradigm. Somewhere in my file room — in binders — is every issue from 1985-2001.
Karen, I just downloaded Katie’s work last week, and it’s on my reading list for holiday break. It would be heresy to have a discussion of measurement without mentioning her name. Of course, if Katie stops by this post, she might well show us how little the rest of us really know! For those who aren’t subscribed, you need to be taking the feed from Katie’s Measurement Blog: http://kdpaine.blogs.com/
Katie Paine’s e-book (free download) applies the concepts of relationship measurement from Hon and Grunig to actual cases: http://kdpaine.blogs.com/bookblog/files/PaineBook4-25.pdf
Hey Bill; Thanks for pointing me to your article. It’s funny that today Geoff Livingston, who’s case study resource at the “Now Is Gone” blog you point to in your post, just today asked people to post about the issue. Funny too that yesterday I taught a PR measurement session in Austin. So it must be fall measurement madness. By th way, I am co-contributor at Now Is Gone, and it was my idea to start the case study resource.
That said, you are absolutely right that many of our case studies don’t point to hard ROI. One of my case studies is in there, and since I wrote it we have tied it to sales, but the other problem is that most clients are not too excited to share their sales numbers in open forums.
So, we have to talk in terms of abstracts of best practices and methods.
And by the way, there are some CEOs that do care about the relationships as much as the sales and reduce costs.
Yesterday, Dell said that their negative comments in the blogosphere have plunged from 49 to 22 percent. I think Michael Dell is pretty happy with that result.
I look forward to getting to know Pat Jackson’s work better, he sounds like my kind of guy.
I added his book to my wishlist at Amazon 🙂