Exploring the core elements needed and depth of analytical knowledge required to lead public relations-oriented research projects
By Natalie Bovair, APR
Recognizing that research is essential to strategic public relations planning, not to mention program evaluation, conventional wisdom would have it that public relations professionals—at least those of the strategic management variety—were taught or developed strong research skills.
Herein lies the problem: The majority of PR practitioners lack the depth of investigative experience or the knowledge of sampling and statistical theory necessary to lead advanced research projects.
Five common research inadequacies
1. A common example is the failure to use a random sampling method, which introduces errors in results. A similar thing happens when research is conducted on a sub-population and then assumed to represent the broader majority. The Twitterverse, for example, is not reliable as a litmus test for public sentiment. Why? According to the Pew Research Center, this population tends to hold more liberal views than the general population.
(See, Twitter reaction to events often at odds with overall public opinion.)
2. Ensuring statistically valid results is another universal concern. For example, if undertaking a national survey in Canada (population 35 million), one needs precisely 384 respondents in order to achieve a 95 per cent confidence level and confidence interval of +/- 5 per cent. (See, Determining sample size: How to ensure you get the correct sample size.) Next, if the organization or individual wants to be able to compare results within this survey—along demographic or geographic lines for example—minimum response rates are needed to make comparisons possible. In fact, a sample size of 384 respondents in all fields being compared is required to keep the confidence level consistently high.
3. Too often, an inadequacy with a major impact happens at the very beginning: Surveys need a proper hypothesis and an effective introduction. Think about the surveys you have completed in recent months or years, particularly how many questionnaires had an effective introduction to orient respondents and achieve informed consent.
Core elements needed:
- a statement of purpose
- an indication of how the results will be used
- a guarantee of anonymity; and
- an accurate estimate of the time needed to complete the questionnaire
4. Another area that results in research inaccuracy relates to the framing of questions, where answers by the respondent are impacted.
Ambiguous, leading, loaded and double-barreled questions are among the more common shortcomings, particularly if the person writing the research questions has a limited understanding of survey design. Respondent error can also be caused by questions that make serious demands on memory. Example: Do you recall the brand of beer advertised during the Super Bowl?
5. Which leads to the final error of thought in this list: Let’s also name pseudo-research as a problem, i.e., conducting research to bolster a point of view.
Zikmund and Babin (see, Essentials of Marketing Research) warn against such pseudo-research, advising the practitioner to, “Walk away from the project if it appears that management strongly desires a predetermined opinion.”
Making the case for a helping hand for public relations research
Back in 2009 on this PR Conversations blog, Dr. David Michaelson proposed 10 best practices in public relations research (see, What are best practices in public relations research, really?); a foundation that the Institute for Public Relations has since advanced.
This is excellent information, but the question remains as to whether the average PR practitioner is any better equipped to lead research projects than before?
Following are some personal examples.
Against my better judgment (under orders from a staff member more senior to me), I once presented a poorly designed survey to a pharmaceutical company client. The client recognized the inadequacies of the research and rightly tore a strip off of me…nearly firing the agency over the incident. Happily for everyone involved, the problem was remedied when a certified research professional redid the work.
More recently, I argued for respondent anonymity in a research project undertaken by a public relations association. I lost the argument, despite providing evidence that informed consent and respondent anonymity are accepted norms in ethical research practice:
“If the Respondent…has given permission for data to be passed on in a form which allows that Respondent to be identified personally: The Respondent must first have been told to whom the information would be supplied and the purposes for which it will be used”
(Reference: Code of conduct and good practice for members of the Marketing Research and Intelligence Association.)
Obviously it’s unethical for an organization to conduct research under the guise of confidentiality, if the intent is to identify who said what; however, I’m sorry to tell readers of PR Conversations that it can, and does, happen, when a bunch of people who consider themselves sufficiently experienced, but don’t have actual research backgrounds, design the survey.
If not you, then who is qualified?
If your research project is centred on public relations practice, you might consider approaching an academic within our discipline.
Look for an academic (or hybrid academic/practitioner) who is conducting research (and publishing it) on a regular and ongoing basis. Individuals who have achieved a master’s degree level of education will have had a proper introduction to research, but it’s unlikely they live and breathe research design and statistical analysis.
The suggestion is to engage a practice-area expert at the PhD graduate level, preferably someone who has demonstrated a varied range of research experience.
Research companies also can be consulted, but play close attention to the credentials of your research counsel. For example, in Canada individuals who hold the Certified Marketing Research Professional designation are a reliable choice.
Statistically speaking, has the time come to embrace math?
Public relations research isn’t all dreadful, everywhere, of course; worthy efforts are being made to elevate the role and practice of research across the communication discipline.
According to the USC Annenberg GAP VIII results (USA), 71.4 per cent of the PR professionals surveyed “strongly agree that there is a need for communication/public relations professionals who can interpret data and use it to plan and evaluate programs. Coupled with the increases in percentage of total budget allocated to evaluation seen over multiple GAP studies, these data reflect the growing importance of analytics and accountability in the practice.”
All of which leaves us facing the reality that, statistically speaking, PR professionals can no longer avoid math.
Either that or they need to seek professional research help.
_____________________________________________________________________________________
Natalie Bovair, APR, owns and operates a solo PR practice in Ottawa.
Over the course of her 25-year career, she has developed specialized skills in healthcare communications and association management. More recently she branched out to teaching public relations to post-graduate students. Her interest in professional issues and the advancement of public relations practice in Canada dates back to her involvement on the student steering committee of the Canadian Public Relations Society (CPRS) in Toronto (1987). She is currently a member of the CPRS National membership committee, and chair of accreditation with her home society, CPRS Ottawa-Gatineau. Natalie has earned four national awards for public relations work. She is accredited in public relations (APR), holds a BBA degree, and earned an advanced diploma in public relations, prior to first entering practice.
For more information, check out Natalie Bovair’s website or LinkedIn company page or follow her personal Twitter or LinkedIn accounts.
Her first contribution to PR Conversations was Two glaring gaps in an otherwise glowing review of senior PR practice in Canada.
_____________________________________________________________________________________
Images supplied by Natalie Bovair (two purchased from Dollar Photo Club and the third copyright-free one from Pew Research).
Great info, Natalie. And don’t beat yourself up over the poorly designed survey, if your boss said it was good and it stunk, then let that be their problem.
You’ve certainly sparked a fascinating and important conversation, which was your point. I would add to those voices who called for a blended approach that includes quantitative and qualitative methods. Much of our success in PR hinges on our ability to use communication to create relationships and shared understanding. Both relationships and meaning are complex areas that can often be clarified best by a blended approach that has both rigour and sensitivity to nuance. As a PR practitioner for more than 30 years, I ran both a PR agency and a subsidiary research firm. Now, as a professor, I teach courses in both areas. It’s a natural combination that helps generate stronger results.
Bernard, thank you for bringing our attention to the value of both qualitative and quantitative research in public relations practice.
I have also observed the routine use of qualitative information by PR professionals, often in order to understand the context for communication. By engaging a representative group of a target audience, for example, it is possible to create meaningful characterizations and interpretations of a situation. This sort of qualitative research is often exploratory and may involve small group discussion, idea generation, concept testing, or probing techniques. I can scarcely think of a major communications initiative that is launched without qualitative research input. Can you?
Quantitative research, on the other hand, is numbers based. Here, PR professionals are able to measure the absolute degree of message acceptance and adoption by a target audience, or determine the number of target audience members who receive, share and/or respond to social media messages, for example.
I think you are right about the need for sensitivity to nuance in communications research–and in PR practice–so long as our practice decisions are supported by research rather than gut instinct alone?
There is also the need to demonstrate a strong relationship between the communications actions taken and the desired organizational outcomes–something that can not be achieved without making research part of every major public relations program.
Natalie – thank you for this post which raises an important issue regarding undertaking research. I think we need to be very clear about the rationale for employing research and indeed, then whether quantitative research and the method of utilising a survey is indeed the best option (and if so, then this leads onto other decisions regarding how and when, as well as the details of population, sample size, question format, etc).
I certainly support your call for undertaking research – and having the mathematical/statistical skills required for analysis (and I don’t just mean total percentages and a basic average) – and argued in The Public Relations Strategic Toolkit for practitioners to implement a Public Relations Information System Management (PRISM) approach as part of adopting knowledge management in underpinning the function and its operation.
My belief is that we need to take a proactive, listening approach to gather information from inside and outside the organisation, draw on existing (secondary) research as well as being able to undertake formal and informal primary research using quantitative and qualitative methods as appropriate to achieve specific research aims (necessitating detailed objectives).
We also need to consider ongoing as well as ad hoc research and focus this on four core areas:
(1) the organisation itself
(2) issues (opportunities/threats)
(3) those who affect or are affected – and influencers on these – in terms of knowledge, attitudes and behaviour
(4) the broader context/society which includes the competitive environment. I’d add in also a time dimension – so understanding the historical context, contemporary situation and forecasting in terms of possible future developments.
Through a rigorous process of analysis (for qualitative as well as quantitative data), we need to be able to provide intelligence and insight to explain and justify our recommendations and guide decision-making.
All of this needs to be undertaken within a clear ethical framework as you indicate. This means understanding the legislative and regulatory context. For example, where you’ve discussed anonymity etc, in the UK, the Data Protection Act covers collection and use of data (https://www.gov.uk/data-protection/the-data-protection-act).
Whilst I concur with you that an understanding and ability to conduct (and analyse) research is important for PR practitioners, it is vital that we know when to call in the experts. Yes, as you say these may be in the academic field, but also we should be familiar with the professional research field. I included a Research Proposal Checklist in the appendices of the above Toolkit book to help cover the main considerations and how to brief a research company.
I appreciate that often the purpose of research seems to be to generate media/public interest, but that’s a rather superficial attitude and is one that is partially responsible for the flaky reputation of PR in research terms. I’d like to see not only much more transparency in revealing who is behind research, but its methodology and how data has been analysed. Sadly there is still far too much reliance on the dodgy survey by PR practitioners and the media to create headlines and stories. Research – and PR practice – deserves much better than this.
Heather, your reply gets to the very heart of this matter.
Where I had attempted to persuade public relations practitioners of the need to retain research council, you have articulated why research is essential to public relations practice.
I agree that PR professionals need a general knowledge of primary and secondary forms of research, and must be acquainted with academic and professional researchers. It is after all the PR professional who needs to make the call for research when it is needed and who needs to know what sort of research is necessary given the circumstances.
Your Public Relations Information System Management (PRISM) approach certainly appears to have merit as do the four noted areas for research focus.
I really must get my hands on “The Public Relations Strategic Toolkit” as a desk reference (SEE http://bit.ly/1AB526U)!
Great feedback on the value of valid and reliable data. Cronbach’s alpha should weed out loaded questions, decent practitioners are available at many universities and should be used if the account manager doesn’t have the skillset or time. Cost are higher, but the data is worth shouting about and can stand the rigors of criticism. My concern is the aging of data out there. I’ve put out a few news making stats in my day and am alarmed to see them continue to be reported as if they are “current” news. Given they measured knowledge and behavior some years back, even I would hesitate to use my own research today!
{Apologies on cross-posting. This first appeared in LinkedIn comments.]
Thank you for the pointer to Cronbach’s alpha test of reliability as a means to weed out bias in some questions. I’m afraid that knowledge of this and other measures of research rigor (SEE A primer of the validity of assessment instruments, http://1.usa.gov/1AAV1X9) are largely unknown to practitioners.
Natalie, I fully endorse all your points.
I began using research in my PR agency in the mid seventies.
At the beginning mostly to capture favourable opinions on issues that strengthened my clients’ arguments to release to media or to politicians. This proved very easy and questions would often be tweaked to receive predetermined results.
Then in our market everybody else began doing this and in seeing so many arguments (not in line with those of my clients) being given credibility by media and politicians I quickly realised that this was not an effective strategy so I stopped using this ‘manipulative tool’ in the late seventies.
I began to use more serious researchers and learned how to interpret the results they collected.
It seems to me today that the most serious issue is for researchers to get their act together and oversee and expose the work of their less respectable colleagues while the Institute for Public Relations has made a major dent into corporate PR departments and serious agencies.
The next step for both groups is to agree that interviewees be always informed of reasons for interview, party commissioning the research and be given access to interpretation process.
Toni, I”ve also seen loaded consumer surveys used as a tool to grab a headline. I appreciate the point you make about the value in disclosing the study sponsor to respondents and making result available. These days media won’t glance sideways at research results without full access to the questionnaire and detailed analysis. It’s useful that our fifth estate also demands a high standard.
You are quite right to point out how unisexy it is to talk about research design and statistical analysis, Mark. Precisely why I thought I’d try to start a conversation here. As a humble practitioner myself, I suppose I have learned enough about research to develop a deep appreciation for what I DON’T know on the subject.
Natalie,
Thanks for reinforcing the essentials of survey research for public relations. As a researcher myself, one challenge is to communicate what’s required but in a language that’s accessible and actionable. Beyond “no ad values,” most PR people glaze over in the detail. Your commitment to professional research standards is admirable. But the profession –which is, after all, rooted in the social sciences — dissociates itself from the science of PR. All we can do is to continue to reinforce key themes as you have.
Mark
Interesting call to basic skills and sound advice Natalie. There are always temptations to cut corners or design pseudo research to illicit certain responses.
Now more than ever public relations practitioners need to base their strategies on sound research.
Yes. It’s really exciting to see how many PR pros place a high value on research to support strategic action. I’m hoping to draw attention here to the need for rigour in the process. And that’s something easily attainable by making the consultation tent bigger, to include ‘research nerds’l