Models of Evaluation

Just take a look to the most popular of the existing models of evaluation proposed by PR scholars. Cutlip et. al’s evaluation model (1985) is known as PII (Preparation, Implementation, Impact). The first step of preparation assesses the adequacy of background information, the appropriateness and quality of message. Implementation examines the number of messages sent to the media and who received them. Finally, impact evaluates the changes in opinion, attitudes and behaviours.

MacNamara’s Macro Model (1992) represents public relations programmes in a pyramidal form with three layers: inputs, outputs and results. The difference to the previous model is that it proposes methodologies, which can be applied to each of the steps, in order to develop a complete process. “MacNamara says that it presents a practical model for planning and managing evaluation of public relations and recognizes communication as a multi-step process” (cited in Kitchen, 1997, p.292).

Lindenmann’s yardstick (1993) also sets a three-level approach, with objectives set beforehand. The first basic level measures outputs, such as media placements. The second intermediate level judges the number of messages received by the target audience, and the last advanced level examines outcomes, such as attitude changes (Theaker, 2004, p.302).

Principles of Evaluation

I believe that the seven principles of evaluation according to Noble (1994) in his paper “Towards an inclusive evaluation methodology” (Corporate Communications Journal, 4(1), pp. 14-23) are really useful for practitioners to understand the importance of evaluating techniques.
According to Noble, “evaluation is:
· Research: its purpose is to inform and clarify.
· Looks both ways: evaluation is a proactive forward-looking activity and, also, a reviewing backward-looking one.
· User- and situation-dependent: it is undertaken according to the objectives and criteria that are relevant to the organization and campaign concerned.
· Short term: campaign or project based.
· Long term: at a broader, strategic level, e.g. corporate reputation.
· Comparative: it frequently makes no absolute judgements but instead draws comparative conclusions.
· Multi-faceted: it is established as a multi-step process with a range of different evaluation methodologies required at each step.”

Surveys' Results

A number of surveys have been conducted the past years, which indicate unfortunately an inconsistency between what practitioners believe about evaluation and what they put into practice. Tom Watson, in an 1992 study in the UK, found that 75% of respondents spent under 5% of their total budget on evaluation, and that the two main methods used were monitoring (not evaluating) press cuttings and intuition. Moreover, the results of an IPRA (International Public Relations Association) survey in 1994 indicated that almost 90% of the IPRA members recognized evaluation as necessary, and 95% agreed that it is more talked about than done. Finally, 31% felt that trying to measure precisely is next to impossible. You can find this information in the IPRA’s Evaluation Gold Paper.

Obstacles to Evaluation

PR practitioners claim that it is difficult to overcome important barriers to use evaluating methods. The most common are:
- lack of time,
- lack of money,
- lack of knowledge and experience in using research techniques and doubts about the process,
- lack of personnel and finally,
- fear of proving non-efficient.

These obstacles can be surpassed, if practitioners study more research methods and techniques. White (1991) in his book How to Understand and Manage Public Relations (p.147) alleges that “Research is the basis of evaluation in public relations practice, contributing to programme development (as formative research), to programme refinement (as diagnostic research) and to assessments of programme effectiveness (as evaluation research)”.

It’s high time gut feeling and AVE was extinct!

For years, the most common practices of evaluation have been the measurement of column inches of press cuttings and other observational and experiential methods based mostly on experience. “Gut feeling” was thought to be an acceptable criterion of whether a campaign has been successful. It was high time, pr practitioners realized that these methods were nor reliable or objective. As a consequence, there was a shift towards more professionalism and practitioners sought to find more scientific methods of evaluation, since 1990 approximately. Still 15 years later, though there has been progress and new models of evaluation have been proposed, numbers contradict what is actually said among PR scholars and practitioners.

Nevertheless, Advertising Value Equivalents (AVE), where an advertising space value is given to media coverage, is still being used as evaluation method from a considerable amount of practitioners, though there is no indication of its validity. Wilcox et al. (1992) describe AVE vividly as “a bit like comparing apples and oranges”.

Quantify Results!

It is unfortunately true that the real value of public relations and its contribution to the total strategy of an organization will only be acknowledged when its results can be quantified and evidence can be presented to the top management. Roger Haywood in his book All about Public Relations (1991, p.323) notes that frequent measurable factors include: budget (to complete activity within the agreed sum), awareness for the organization, possible shift in attitude, media coverage, response generated by the campaign and changes in sales volumes or prices.

Setting Objectives...

A prerequisite for evaluating a public relations programme is to have established first a set of measurable objectives. It is clear that without setting SMART (Specific, Measurable, Achievable, Realistic, Timely) objectives, it is not feasible to measure and assess if these objectives were achieved. The reply of Noble (1999) to this is not really a favoring one: “In an ideal world, the setting of specific, quantified and measurable objectives would indeed be the panacea for effective evaluation. However, public relations is rarely - if ever - able to achieve substantive objectives by itself”.

But, what is evaluation?

At this point, I think it is useful to define the term evaluation. Grunig and Hunt (1984) in the book Managing Public Relations (p. 183) distinguished between outcome evaluation (measures the effects of PR programmes) and process evaluation (examines effective administration of the programme). On the other hand, according to Watson in the book Public Relations: Principles and Practice (Kitchen, 1997, p.284), “definitions of evaluation can fall into three groups: the commercial, which is just a justification of budget spend, simple-effectiveness, which asks whether the programme has worked in terms of output, and objectives-effectiveness, which judges programmes in terms of meeting objectives and creation of desired effects”.

Evaluation in Public Relations: Feasible or not?

A current trend in public communication and public relations, and for sure an issue of considerable concern and controversy, is the attempt to measure the effectiveness of public relations. Over the years public relations practitioners and scholars are striving to find ways to evaluate the contribution of public relations programmes. Many comment that measuring the effectiveness of public relations is like trying to find the Holy Grail. There was and still is a lot of discussion about evaluation in public relations. Besides, in recent years, management demands greater accountability from public relations, as far as its contribution to the bottom line is concerned.

Assignment on current trends in PC and PR

This blog is created as an assignment for my MA in Public Communication and Public Relations. My name is Afroditi and I am from Greece. The last 3 months I live in London, where I am studying at the University of Westminster. Our assignment for Christmas according to Simon's brief is the following:

We want each of you to create an individual blog - a personal web log - which explores a current develpoment in the field of public communication and public relations. It should provide evidence of original research and your own arguments and conclusions.

The subject area is deliberately broad. Current trends in PC/PR include aspects of many of the topics we've looked at this term and much more besides. Please don't try to cover everything - choose an area of PC/PR which interests you, but make sure your work is distinct from other assignments you have undertaken (eg your seminar paper).