I always planned to use real estate space on our collective PR Conversations platform to spotlight some exemplary Canadian PR practitioners. When it comes to an overview of the Canadian measurement scene, few can match the depth and breadth of knowledge of Alan Chumley. Instead of an interview, I offered Alan a guest post, where he could write (in his own words) about measurement in Canada. So, gentle readers, I present to you Alan Chumley, who very much welcomes comments and feedback from our international audience of PR academics and practitioners, colleagues and friends. JG
* * *
I was thankful and honoured when PR Conversations invited this Measurement PRoponent / PRomulgator to opine on the current state of the art (or science?) of measurement in Canada. While it’s a topic that could easily spawn a healthy hundred-plus-page graduate thesis, the following is a reasonably succinct, high-level, non-exhaustive horizon scan of measurement in Canada, broken up into bite-sized morsels to cover the following topics: industry associations; Canadian vendors; U.S vendor spillover; the client context and the spectrum of “outs”; standards, patterns ‘n packages; the agency sphere; homegrown, thought leadership on measurement writ large; Canadian measurement events and conferences; education; awards; the wrap; and the ask.
Many Canadian practitioners look south of the border to the Institute for Public Relations’ Measurement Commission for example, for institutionalized thought leadership on measurement and across the pond to the less prolific, U.K.-based-but-global Association for Measurement and Evaluation of Communication (AMEC). (I’m a booster and disciple of both.)
Canadian associations, perhaps most notably the Canadian Public Relations Society, plus the Canadian Council of Public Relations Firms and the Toronto (and other Canadian) chapters of the International Association of Business Communicators have played a role in advancing the measurement discourse in Canada. They’ve done so through hosting events and through endorsing methodologies.
Case in point, the Canadian Public Relations Society’s measurement committee rolled out its Media Relations Ratings Points (MRP) in April 2006. Endorsed by both IABC/Toronto and the newly minted Canadian Council of Public Relations Firms* it was launched with great fanfare at a very well-attended event, and the method has since seen a fairly healthy take rate. The CPRS Measurement Commission* apparently worked for three years on MRP. They, along with NewsCanada, are to be congratulated for recognizing and working toward solving a distinct need in the Canadian marketplace with, at least, a simple, cheap editorial scoring system for, largely, MarComm-driven coverage. It also got Canadian practitioners en masse talking about measurement again; at least now we’re all drawing from data sources consistently. However, even by the admission of the measurement committee, it’s not without its limitations. It works reasonably well in a very specific context for which it seems to have been designed, beyond which one would still need to look at undertaking additional research if one wanted to go deeper than an 80 per cent score and a simple CPM calculation…and we all know we should be doing that.
Picking up where MRP leaves off, Cormex Research has a healthy grip on the Canadian media content measurement and analysis market. The other key Canadian player, Ottawa-based MediaMiser, uses a hybrid human and automated system for near real-time analysis and reporting. They offer slightly different services, tailored for different client needs. I’ve seen colleagues and clients use MRP in concert with deeper content-analysis and data-cross tabulation.
U.S vendor spillover
MB Precis** completes the odd project—ranging from content analysis to full-impact studies—in Canada, but based out of its offices in the U.S. Similarly, Carma, operating under Montreal-based Leger Marketing in Canada is an occasional “also ran.” The organization formerly known as Delahaye-turned-Observer Group-turned Cision owns a media monitoring arm in Canada, but little of the type of measurement that Delahaye does in the U.S. has crept north of the border.
While I’m encouraged by the amount of in-depth media content analysis that’s being undertaken, at the end of the day it remains a measure of output until—or unless–it’s linked with other sources of data such as polling data. I don’t see much of that happening. Also, I don’t imagine there’s much in the way of Market Mix Modeling or Six Sigma (specific to a communications context) going on in Canada.
The client context and the spectrum of “outs”
In 2004, Watson and Simmons published a study that, among other notables, pointed out that 89 per cent of practitioners are measuring output, but only 32 per cent are measuring bottom-line business or organizational impact. (I suspect this later number to be even lower in Canada). There are several other major studies out there before and since then have come up with similar proportions; however, I suppose the key point here is that, with well-earned respect to notable exceptions, and while there is some excellent work going on in Canada, we continue to be hampered by an output measurement mindset and are challenged with moving people to outtake, outcome and outgrowth. (Perhaps paradoxically, but encouragingly, Canadians have largely rejected the use of “advertising equivalency” numbers). The output-outgrowth war is a battle we measurement mercenaries fight daily. We know there’s more to communications than media relations. It’s about relationships, reputation and everything that’s done to drive and manage both. All of which can and should be measured. I can’t tell you how many times I’ve had a conversation where I’ve asked a client or a prospect, “Did you know that we can measure x?” The result is overhead light bulbs and raised brows at pitch time, but knee-jerk reaction come execution and budget time. An issue not exclusive to Canada, I suspect.
Standards, patterns ‘n packages
I’ve noticed, particularly but not exclusively on the client side, that there’s a tendency out there to want to (over)simplify measurement solutions into patterns and packages such that a client of a certain size with a certain need could be slotted into a bronze, silver, gold or platinum measurement package. The easy answer isn’t always the right one in measurement any more than it is in building one communications plan to suit all client needs (not to mention their reporting structure, operating environment, management style, communications department culture, PR practitioners’ roles, their research orientations and the prevailing model of PR resident in the organization). I suspect I’m preaching to the converted, but, I evangelize to account teams internally and clients and prospects externally that each measurement program must be built to meet specific needs. Adhere to an industry standard? Heck no. There isn’t one and nor should there ever be. Regard industry best practices (such as those that come out of the Institute for Public Relations’ Measurement Commission)? Absolutely. It’s a tough sell, but something we need to continue to promote.
The agency sphere
Canadian agencies just don’t have nearly the research and measurement resources that our brethren in the U.S. do. For example, in Canada, with the exception of Doris Juergens, partner and senior vice president of National PR’s Research, Information Services and Training (who also sits on the CPRS measurement committee) and me at Hill & Knowlton Canada, you’d be hard pressed to find another senior-level position that is devoted exclusively to measurement and/or research and measurement; roles that we see more commonly in the U.S. It’s nothing to do with how measurement is regarded in this country. It’s more to do with what every Canadian organization that also has a U.S. counterpart suffers from. The old order of magnitude issue where one can typically expect to knock at least a zero off the client’s budget.
Homegrown thought leadership on measurement writ large
In my view the list of Canadian measurement thought leaders is limited to Cormex’s Andrew Laing, National’s Doris Juergens, Tudor Williams, and IPR measurement commission member Fraser Likely; however, none have yet emerged nearly as visible and vocal as is KD Paine in the U.S. and globally.
Canadian measurement events and conferences
You’ll often see some of these thought leaders presenting at events with a measurement focus. Though they certainly don’t compare to the Institute for PR’s annual Measurement Summit, there is the odd conference dedicated specifically to measurement, including the 10th Annual Communications Performance Measurement conference. Generally, though, measurement is relegated to a one-hour session at the end of a conference with a broader context such as media relations or government communications. It’s wonderful that measurement is included in these events…but they are usually scheduled at the end of the conference when many attendees are zipping up their bags and dashing to catch a flight. A shame.
Education has an important role to play in how measurement is regarded in this (and in any, I suspect) country. Unlike many other markets, undergraduate PR education has, with very few exceptions, been almost exclusively relegated to strictly vocational training at Canadian community colleges. While there’s a market and a place for that type of program, I fear it tends to marginalize how research and measurement is taught, if it is at all. Based on my admittedly anecdotal research of most of the major undergrad-level PR programs in this country, very few have a stand-alone course dedicated to research and or measurement. If courses specific to fundraising, photography and event management can make the curricular cut, then surely there’s a place for research and measurement.
I recently had the pleasure of judging IABC/Toronto’s OVATION Awards submissions. For the most part the quality and creativity of the projects submitted was top calibre. What was disappointing, though, was how the majority of the projects measured success. For example, if an objective involved raising awareness, then there better be a measure to gage it. Generally, there wasn’t. In my view there’s a need to call for:
1) More rigorous measurement criteria against which to judge submissions.
2) Canadian awards specific to measurement, similar to the IPR’s Jack Felton Golden Ruler award, and AMEC’s awards, a topic that I’ve written about previously.
3) More Canadian submissions to the Golden Ruler. The behaviour of raising our measurement game needs to have more incentives and be adequately rewarded.
It’s only through thought leadership, conferences, awards and increased academic curriculum attention, plus augmented involvement from industry associations and a collective dialogue among stakeholders—including academics, students, vendors, associations, and both agency and client-side practitioners—that we will continue to productively push the measurement agenda forward in Canada. It’s progressed much in the last 15 years, but there’s a long and winding road ahead.
While these may be issues that are sources of similar concern in other regions of the world, I wonder: To what extend do PR Conversation readers feel that they are they shared and/or more or less acute in different countries?
Director of Measurement, Hill & Knowlton Canada
*Disclaimer: my employer, Hill & Knowlton Canada, is not currently a member of the Canadian Council of Public Relations Firms or the CPRS measurement commission.
**MB Precis and my employer Hill & Knowlton are both owned by the same parent: WPP Group.
For more Alan Chumley:
Blog posts on Alan Chumley opines on “the current state of the art (science?) of measurement in Canada: