Future Challenges

It is increasingly acknowledged that the evaluation of public relations programmes requires a mix of techniques. A set of mutually consistent methodologies needs to be developed, from which practitioners can choose according to the circumstances. Public Relations evaluation, in fact, requires a more sophisticated analysis. As Cutlip et al. note “The most common error in programme evaluation is substituting measures from one level for those at another level”. The challenge is to establish and define the range of tools that will form the public relations professional’s future evaluation toolkit. Quantitative and qualitative methods should be used in combination and provide valuable insights into PR results. Public relations practitioners need to adopt a progressive, consecutive approach to measurement and evaluation and develop the necessary skills for collecting data and conducting appropriate research. Standardizing evaluation methods remains a problematic issue in the industry, mainly due to the inability of PR practitioners and scholars to reach to an agreement on how it could be effectively applied. Evaluation is and will continue to be a “hot topic” in public relations theory and practice, until generally accepted ways are found to finally provide evidence of PR’s contribution to results.

Steps to success

I believe that the first step to successful implementation of evaluation is the common understanding and acceptance of the terminology of evaluation. The terms: outputs, outtakes and outcomes should have the same meaning for everyone. Output is what PR has delivered, e.g. the number of press cuttings. Outtake is the degree to which an audience is aware of and has understood the message. Outcome is the change in people’s opinions, attitudes and behaviours.
Pr practitioners should also realize that they are dependent upon the credibility of the top management. PR should be perceived as effective and its efforts to be seen as contributing to the fulfillment of management goals. Tixier (1995) noted for PR: “To exist and survive, it is important to prove that it is both useful and beneficial”. For this reason more weight should be laid on the use of evaluating techniques. Setting specific and measurable objectives can be really helpful to the process. It must be underlined that practitioners should do evaluative research not only after the end of the campaign, but also during planning. This could give them an important advantage. The view about evaluation should change. Evaluation is meant to be continuous. Tactics and plans must be frequently measured, and according to the feedback reconsidered and changed.

PR Toolkit

As I mentioned yesterday, the IPR along with the PRCA, AMEC and PR week created in 1999 the PR Toolkit, a framework of measurable objectives to encourage a wide audience to rethink their planning and evaluation techniques (written by Michael Fairchild). As noted in PR Week by Kate Nicholas (30/04/1999), “this practical guide to R&E moves the debate beyond why measure, to how to measure. The toolkit recognizes the diversity of the disciplines known generically as PR, the range of audiences with which one interacts and the way in which PR is expected to interact with other disciplines”. The Toolkit suggests a sequence of five steps as part of the planning process:
1. Audit of existing communications and background,
2. Constructing of objectives,
3. Strategy and planning,
4. Ongoing Measurement and
5. Results and Evaluation.
The Toolkit puts emphasis on the fact that there is no single industry-wide-measure, but there is a wide variety of tools available. As Lindenmann (1993) already noticed some years ago, “there is no one simplistic method for measuring PR effectiveness…an array of different tools and techniques is required to properly assess PR impact”.
The disappointing thing is that 5 years after, there is no actual change in the attitude of the practitioners towards evaluation. A PR Week 2004 Survey showed that 68% of companies spend 3% or less of their PR budget on evaluation and only 4% of respondents allocate at least 10%. Furthermore, 46% of respondents said they rely on gut feel to assess campaigns’ effectiveness – a rise of 13% over the past five years. Meanwhile, 30% of respondents said they use AVE, up from 9% in 1999. The results of the survey are not exactly encouraging. That’s why the trend for understanding how to evaluate and why, and establishing a standard evaluation system is still on.

Proof Campaign

PR Week magazine has conducted also many surveys on evaluation. Its Proof Campaign, launched in 1998, is one of the attempts made from many associations to establish commonly acceptable methods of evaluating PR results. Aim of the campaign was to encourage companies to allocate 10% of their budgets to research and evaluation. “The IPR, the PRCA, the Public Relations Standards Forum (PRSF) and the Association of Media Evaluation Companies (AMEC) agreed to join forces to produce definitive best practice guidelines for the use of research and evaluation within the industry”, which would take the form of a toolkit (PR Week, Suzan, Leavy, 6/11/1998)
In 1999, the Proof survey was the first to compare the views of agency, in-house and public sector practitioners. “PR Week decided to try to establish a more detailed picture, not only of how practitioners working in different PR disciplines view, but also how they use, the different R&E methodologies available to them”. According to the survey, a total of 20% of respondents said that the success of their efforts could not be evaluated. “The most commonly used method across all disciplines was media content analysis and press cuttings, closely followed by media reach/OTS. It is interesting to note that 61% of all respondents said they have used the technique as a means of planning and evaluating their campaigns, while just 34% regard it as an effective tool” (PR Week, Cowlett and Nicholas, 12/03/1999). The results of this survey depict the real attitude of practitioners towards evaluation.
This lack of standardization has led many PR consultancies and in-house PR departments to create their own evaluation tools, like for example Kaizo consultancy or Hewlett Packard, which has even published a report “Measuring Media Relations”, containing 148 ways in which PR can be measured.

Models of Evaluation

Just take a look to the most popular of the existing models of evaluation proposed by PR scholars. Cutlip et. al’s evaluation model (1985) is known as PII (Preparation, Implementation, Impact). The first step of preparation assesses the adequacy of background information, the appropriateness and quality of message. Implementation examines the number of messages sent to the media and who received them. Finally, impact evaluates the changes in opinion, attitudes and behaviours.

MacNamara’s Macro Model (1992) represents public relations programmes in a pyramidal form with three layers: inputs, outputs and results. The difference to the previous model is that it proposes methodologies, which can be applied to each of the steps, in order to develop a complete process. “MacNamara says that it presents a practical model for planning and managing evaluation of public relations and recognizes communication as a multi-step process” (cited in Kitchen, 1997, p.292).

Lindenmann’s yardstick (1993) also sets a three-level approach, with objectives set beforehand. The first basic level measures outputs, such as media placements. The second intermediate level judges the number of messages received by the target audience, and the last advanced level examines outcomes, such as attitude changes (Theaker, 2004, p.302).

Principles of Evaluation

I believe that the seven principles of evaluation according to Noble (1994) in his paper “Towards an inclusive evaluation methodology” (Corporate Communications Journal, 4(1), pp. 14-23) are really useful for practitioners to understand the importance of evaluating techniques.
According to Noble, “evaluation is:
· Research: its purpose is to inform and clarify.
· Looks both ways: evaluation is a proactive forward-looking activity and, also, a reviewing backward-looking one.
· User- and situation-dependent: it is undertaken according to the objectives and criteria that are relevant to the organization and campaign concerned.
· Short term: campaign or project based.
· Long term: at a broader, strategic level, e.g. corporate reputation.
· Comparative: it frequently makes no absolute judgements but instead draws comparative conclusions.
· Multi-faceted: it is established as a multi-step process with a range of different evaluation methodologies required at each step.”

Surveys' Results

A number of surveys have been conducted the past years, which indicate unfortunately an inconsistency between what practitioners believe about evaluation and what they put into practice. Tom Watson, in an 1992 study in the UK, found that 75% of respondents spent under 5% of their total budget on evaluation, and that the two main methods used were monitoring (not evaluating) press cuttings and intuition. Moreover, the results of an IPRA (International Public Relations Association) survey in 1994 indicated that almost 90% of the IPRA members recognized evaluation as necessary, and 95% agreed that it is more talked about than done. Finally, 31% felt that trying to measure precisely is next to impossible. You can find this information in the IPRA’s Evaluation Gold Paper.

Obstacles to Evaluation

PR practitioners claim that it is difficult to overcome important barriers to use evaluating methods. The most common are:
- lack of time,
- lack of money,
- lack of knowledge and experience in using research techniques and doubts about the process,
- lack of personnel and finally,
- fear of proving non-efficient.

These obstacles can be surpassed, if practitioners study more research methods and techniques. White (1991) in his book How to Understand and Manage Public Relations (p.147) alleges that “Research is the basis of evaluation in public relations practice, contributing to programme development (as formative research), to programme refinement (as diagnostic research) and to assessments of programme effectiveness (as evaluation research)”.

It’s high time gut feeling and AVE was extinct!

For years, the most common practices of evaluation have been the measurement of column inches of press cuttings and other observational and experiential methods based mostly on experience. “Gut feeling” was thought to be an acceptable criterion of whether a campaign has been successful. It was high time, pr practitioners realized that these methods were nor reliable or objective. As a consequence, there was a shift towards more professionalism and practitioners sought to find more scientific methods of evaluation, since 1990 approximately. Still 15 years later, though there has been progress and new models of evaluation have been proposed, numbers contradict what is actually said among PR scholars and practitioners.

Nevertheless, Advertising Value Equivalents (AVE), where an advertising space value is given to media coverage, is still being used as evaluation method from a considerable amount of practitioners, though there is no indication of its validity. Wilcox et al. (1992) describe AVE vividly as “a bit like comparing apples and oranges”.

Quantify Results!

It is unfortunately true that the real value of public relations and its contribution to the total strategy of an organization will only be acknowledged when its results can be quantified and evidence can be presented to the top management. Roger Haywood in his book All about Public Relations (1991, p.323) notes that frequent measurable factors include: budget (to complete activity within the agreed sum), awareness for the organization, possible shift in attitude, media coverage, response generated by the campaign and changes in sales volumes or prices.

Setting Objectives...

A prerequisite for evaluating a public relations programme is to have established first a set of measurable objectives. It is clear that without setting SMART (Specific, Measurable, Achievable, Realistic, Timely) objectives, it is not feasible to measure and assess if these objectives were achieved. The reply of Noble (1999) to this is not really a favoring one: “In an ideal world, the setting of specific, quantified and measurable objectives would indeed be the panacea for effective evaluation. However, public relations is rarely - if ever - able to achieve substantive objectives by itself”.

But, what is evaluation?

At this point, I think it is useful to define the term evaluation. Grunig and Hunt (1984) in the book Managing Public Relations (p. 183) distinguished between outcome evaluation (measures the effects of PR programmes) and process evaluation (examines effective administration of the programme). On the other hand, according to Watson in the book Public Relations: Principles and Practice (Kitchen, 1997, p.284), “definitions of evaluation can fall into three groups: the commercial, which is just a justification of budget spend, simple-effectiveness, which asks whether the programme has worked in terms of output, and objectives-effectiveness, which judges programmes in terms of meeting objectives and creation of desired effects”.

Evaluation in Public Relations: Feasible or not?

A current trend in public communication and public relations, and for sure an issue of considerable concern and controversy, is the attempt to measure the effectiveness of public relations. Over the years public relations practitioners and scholars are striving to find ways to evaluate the contribution of public relations programmes. Many comment that measuring the effectiveness of public relations is like trying to find the Holy Grail. There was and still is a lot of discussion about evaluation in public relations. Besides, in recent years, management demands greater accountability from public relations, as far as its contribution to the bottom line is concerned.

Assignment on current trends in PC and PR

This blog is created as an assignment for my MA in Public Communication and Public Relations. My name is Afroditi and I am from Greece. The last 3 months I live in London, where I am studying at the University of Westminster. Our assignment for Christmas according to Simon's brief is the following:

We want each of you to create an individual blog - a personal web log - which explores a current develpoment in the field of public communication and public relations. It should provide evidence of original research and your own arguments and conclusions.

The subject area is deliberately broad. Current trends in PC/PR include aspects of many of the topics we've looked at this term and much more besides. Please don't try to cover everything - choose an area of PC/PR which interests you, but make sure your work is distinct from other assignments you have undertaken (eg your seminar paper).