Research and evaluation are considered by many to be relatively recent concerns of public relations practitioners. Developments include the (recently updated) Barcelona Principles, ‘workflow‘ tools enabling monitoring and analysis of data generated through digital communications, numerous academic research papers and the insight of research firms. Several chapters in the new Future PRoof book, including my own on sustainable professional development, discuss measurement matters such as return on investment. The lack of standardised industry measures is still argued as a weakness of PR, alongside swapping the discredited AVE with “PR Value” (which is rebranding a bogus measure).
My view is that monitoring and evaluation need to be connected to research and objectives to have relevance as measures of public relations performance. So it is interesting to note that Part IV of the 1948 book, Your Public Relations, which we are serialising at PR Conversations, contains just one chapter authored by Dr Claude E. Robinson, President, Opinion Research Corporation, called: How to Use Opinion Surveys in Public Relations.
Born in 1900, Dr Robinson became a pioneer of research. He served in the US Army in World War I, before obtaining Masters and PhD degrees in sociology from Columbia University. He invented a method of measuring listener response to radio broadcasts and published many books before his death in 1961.
The book’s editors, Glenn & Denny Griswold write:
“Probably no man in American has done as much for Dr. Robinson to dramatise the fact that public attitudes can be scientifically measured and that appraising public opinion is the first step in public relations. His Public Opinion Index for Industry provides exhaustive public attitude studies for the guidance of many of our big industrial corporations.”
Dr. Robinson does not disappoint, as he states:
“Research is not an end in itself. It is of no practical value unless it inspires action, guides action, and assays the impact of action. Thus the researcher and public relations expert are collaborators working toward a common goal.”
He advocates “A careful study of public opinion defines the problem. It shows who are the friends and who are the opposition. It reflects the information and misinformation people have, and demonstrates the relations between information and misinformation and attitudes. It identifies the urgent and desires that motivate people. It indicates the positive and negative symbols to which people react.”
Such research is not focused on tactics, but on strategy as if the latter is wrong “even brilliant day-to-day tactics will fail to get results”. Indeed, “with objective data in hand, the public relations man can then go to his superiors and get agreement on the course of action to be followed and secure the budget necessary to carry on the campaign” – an argument that resonates with contemporary PR practice.
In addition to formative research to help guide strategy, Dr. Robinson moves onto pre-testing of public relations materials. In my experience, this stage is one that is more common in advertising and marketing than public relations communications. But “the symbolism chosen for a public relations document may represent perfect clarity to its creator, but be both uninteresting and unintelligible to the reader”. Both “themes and treatment of themes” can be tested to show “what types of stories were most believable”.
Next comes auditing of public relations campaigns. Interestingly Dr. Robinson writes sagely:
“The difficulty heretofore with most attempts to measure public relations impact is that the principals hoped for too much change in public opinion and were disappointed by the actual results. One would not expect to drop a pebble into a pool and see the water level rise three inches. Similarly, an ad or a news article or a booklet cannot be expected to transform opinion overnight. Usually the public changes its views slowly and the engineering of this chance is a long and hard job. Competition for the public’s attention is fierce.”
Referring to the “tracer theory” (rather than the “before and after theory”), Robinson focuses on being able to “trace the penetration of ideas in relation to exposure to media”. Today this is called attribution theory in relation to online marketing (drawing on Heider’s work from 1950s).
Another contemporary idea in the book chapter relates to the importance of being able to draw out the implications of data analysis. Robinson also applies his recommendations to “measuring employee attitudes” and “appraising community opinions”.
Naturally, it is in the practical sections of the chapter where the limitations of Robinson’s approach are revealed. However, the focus on understanding “attitudes, convictions, beliefs and prejudices of individuals” remains true if we are looking at human psychology to understand and drive behaviour.
The only research tools discussed in the chapter are “impressionistic observation” and “opinion sampling”. The first involves “a way to ‘size up a situation’ quickly, with a minimum of toil” but relies on “more skilful, more practiced observers”.
As Robinson explains: “the method of impressionistic observation, however, has grave short-comings. It is frequently erratic, producing grossly inaccurate pictures of public opinion. Impressions involve a large degree of subjectivity. When the observer is emotionally involved, he has trouble weighing his impressions correctly”.
In contrast, whilst stating that “opinion sampling is less flexible than impressionistic observation” and is more labour intensive, “it has a high content of objectivity” and provides outcomes that can be checked. “Like any other scientific process, a sampling operation can be repeated under parallel conditions and produce substantially identical results”.
The chapter goes onto consider the importance of “what to ask? Whom to question? How many to question?” to ensure “technical accuracy” in opinion surveys. Robinson also recommends understanding where people’s opinions are informed, and where they are misinformed. He writes:
“One of the major functions of research is to map out the important areas of ignorance.”
This often seems forgotten in modern studies where results may stated as unquestionable truth, rather than as a source of understanding. Another key research aspect is ensuring “the questioner and the respondent are talking about the same thing”, as there can be variance in “meanings and emotional connotations”. Bias and ambiguity can often be found in modern surveys – indeed, we commonly see research undertaken by PR practitioners solely to generate or drive a story rather than as a source of insight to inform practice or policy.
Interpretation of results is emphasised in the chapter with examples of poorly constructed surveys used to illustrate how “the naive observer could jump to the (wrong) conclusion”. Sampling issues are discussed with matters of representation and statistical validity raised. After presenting some case studies, Dr. Robinson concludes his chapter by discussing costs, arguing:
“A research budget must necessarily bear some relationship to what is spent on public relations, or the importance of the public’s attitude to the company’s welfare. If much public relations effort is being undertaken, then the questions should be, ‘Can we afford not to undertake research to guide us in this program?'”.
Evidence that Dr. Robinson was not alone in his expertise, he recommends additional sources published by his business partner, George Gallup – including the fabulously named: The Quintamensional Plan of Question Design, as well as Hadley Cantril and Blankenship’s “practical and informative guide.”
The ongoing relevance of this chapter continues with the views of the Griswolds who discuss management impressions that public relations work is intangible and difficult to measure. They argue that the development of approaches to measure public attitudes “accurately and scientifically” were being used by many larger corporations (listed by name), and could be adapted to the requirements, and smaller budgets, of “medium-sized and even small corporations”. They conclude their summation with reference to the need to “follow the trend of public opinion an attitude as revealed in a wide variety of continuing studies”.
Although today we have greater familiarity with surveys – including understanding their limitations – than in 1948, the deployment of research is arguably not routine in informing, and evaluating public relations activities. Cost and other constraints that were probably common in the 1940s remain those that I hear today from practitioners as reasons for relying on “impressionistic observation” as Robinson would term it. Our gut instincts are probably no more valid than those of our predecessors. Of course, we may also argue that research does not necessarily result in more effective plans and programmes, but at the least, it may help avoid costly mistakes, and reinforce a professional approach to our craft. Understanding and engaging with people will never be the most exact of sciences – but if we aren’t trying to underpin our work with an evidence base, we cannot be surprised when public relations itself is seen as little more than opinion, even when advocated by experts.
Image via: http://rogerwilkerson.tumblr.com