It’s been an exhausting week here in New Zealand. Most of the population has been staggering round bleary-eyed but elated having sat by TV screens into the small wee hours to watch our football team triumph in the FIFA World Cup. Oh, sorry – didn’t you know we won? Ryan and the boys might be on their way home, but the certainty here is that they were undefeated in the tournament and finished ahead of World Champions Italy.
Of course, the tournament is still in full swing and whoever lifts the trophy yet to be decided, but New Zealand’s outcome – for a country with just 25 professional footballers and ranked 78th – is an emphatic and glorious win, with a proud nation hoping someone will organise a welcome home victory parade. All of which only goes to show why it is essential to understand the difference between measuring outputs and outcomes.
This month saw another cross-border event, with the agreement and publication of the “Barcelona Declaration of Research Principles” at the 2nd European Summit on Measurement. Five global bodies and 200 delegates from 33 countries all voted overwhelmingly to adopt the following principles:
- Goal setting and measurement are fundamental aspects of any PR programmes.
- Media measurement requires quantity and quality – cuttings in themselves are not enough.
- Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity.
- Social media can and should be measured
- Measuring outcomes is preferred to measuring media results
- Business results can and should be measured
- Transparency and Replicability are paramount to sound measurement
While I was delighted to see this agreement finally putting the lid on the unethical tosh and nonsense that is AVE, I was still frustrated to see this list so closely tied to media, when public relations is concerned with so much more. I was also a little concerned to see the list tucked under the heading of ‘research principles’, when research and research methods don’t get a mention. But hey, like NZ qualifying for the World Cup in the first place, it is a beginning.
The steps and stages of public relations and communications evaluation have been around for decades and any practitioner worth their salt has a robust process in place that informs and guides strategy, engagement and relationship building as well as assessing effectiveness against organisational outcomes. Generating coverage centric numerics and retaining a focus on media measurement without moving it into an evaluation process generally leaves people with a list of nice looking but meaningless numbers. After all, a measurement gives you a fixed unit – e.g. how long is a piece of string – but an evaluation provides you with the significance of the outcome – e.g. how well did the string tie my shoes, was it the right length for the job, would something else have been better?
If I look at our football team’s numbers – 1-1, 1-1, 0-0 – it tells me they didn’t win a match. Not a good look in a world-class soccer tournament. But if I evaluate against the desired nationally recognised outcome set before they left, which was to take part in the World Cup Finals and, if possible, get NZ’s first ever point, the numbers reveal the outcome was not only reached, but exceeded. The evaluation – but not the numbers – would also reveal that the team won the collective heart of a previously rugby-obsessed nation, (changed dynamic), increased active participation in soccer to the extent you can’t walk down the street without seeing someone kicking a soccer ball (changed behaviour), and there is no doubt at all here now that New Zealanders can, indeed, play soccer (changed perception).
Pre- and post- research would have provided me with the specific qualitative data I needed to demonstrate the difference made by our participation in the World Cup to the country’s relationship with soccer (social impact). Other relationships will have altered too – education and soccer, sports equipment suppliers and soccer, mainstream media and soccer – all important and measurable relationship strands within the overarching relationship outcome.
Once the boys are home, ongoing research will inform where the next ‘agreed outcome’ should be set. Will we still be happy with ‘win, lose or draw – you make us proud’ or will expectations be much higher based on the changed relationship? And exactly how will that relationship be sustained in the face of increased expectation and demand? Well that’s were we get to work. Such sustenance is an integral part of our role as public relations professionals, but without research holding hands with evaluation and measurement joining the dots between the two, we cannot do it effectively.
The Barcelona Principles are long overdue and are to be welcomed, but personally, I am looking forward to the next declaration in the hope it will really take this essential public relations discipline into the main stadium for the win.
[Note: Access the Barcelona Summit site for details of the Barcelona Declaration of Research Principles.