Evaluation - Did it really work?

1. Evaluation - Did it really work? Effective public relations measurement and evaluation Presented by Dr Tom Watson, Associate Professor in Communication, Charles Sturt University

2. Flying on one instrument?

3. Agenda • The communications process • What is evaluation? • Models of evaluation • Brief case studies

4. How PR is practiced • Grunig’s four models • One-way • Press agentry • Informational • Two-way • Asymmetrical • Symmetrical

5. Communications process Presentation  Attention  Comprehension  Acceptance  Retention  Action - Outcome McGuire (1984)

6. Presentation Attention Comprehension Acceptance Retention Action Output (L1) Out-growth (L2) Outcome (L3) Lindenmann 1993 Communication and evaluation

7. Types of evaluation • Summative – final impact • Formative – “in progress” assessment • Should be integrated into all stages of programs

8. What can be measured? • Success or failure of strategy • Message distribution and reception • Understanding, favourability and acceptance • Trends

9. What else? • Internal • Staff attitudes; value of newsletters, video, team briefings • SWOT • External • Sponsorship impact • Issues management and prediction • Stakeholder attitudes

10. Case studies • PricewaterhouseCoopers (PwC) accountants and biz advisers • DIMIA – Harmony Day • Polestar – major printing group • Volvo - XC90 launch

11. PwC • Why aren’t we getting coverage? – asked the bosses • Merger of PW with C&L not seen as equal – a “takeover” • Change in approach

12. PwC lessons • Charts and graphics improve story acceptance and favourable write-up • CEO in interviews increases favourability – “voice of the organisation” • Benchmark coverage to find “who’s writing about what”

13. DIMIA • Monitored media coverage of events • Moved from celebrity to community basis • More outside metros • Increased participation

14. Polestar • Europe’s biggest print group • Going through major changes • Monitored media coverage • Able to see strategy and tactics working

15. Polestar – mid-2001

16. Polestar – early 2003

17. Volvo – new car launch

18. UK Launch program • Competing against BMW and Mercedes • Strategy was to have continuing campaign instead of launch “blip” • Closely targeted – 25 to 44, mid to high income, degree educated, adventurous • Total of 609,000 people (1% of pop.)

19. PR-led campaign • Media relations drove enquiries to internet and dealers • More interest than cars to sell; All ordered before sale date announced • Advertising ran for one month and cancelled – saving of c.$6m • “PR helped sell the required number of cars”

20. PR works!

21. Evaluation • Three methods of evaluation used: • Responses to internet and then dealers • Reach of coverage to target audience 96% of target group saw it once; 90% twice • Sales

22. Evaluation – your future • Forget the “Holy Grail”, it’s about relationships (Hon & Grunig 1999) • Most PR campaigns don’t influence behaviour. (McCoy & Hargie 2003) • Avoid dead-ends like management language for evaluation, e.g. ROI (Watson 2005)

23. References and Reading • Hon LC & Grunig JE (1999). Guidelines for measuring relationships in public relations. www.instituteforpr.com • Lindenmann, W.K. (1993) An “effectiveness yardstick” to measure public relations success. Public Relations Quarterly, 38 (1) • McCoy M & Hargie (2003) Implications of mass communication theory for asymmetric public relations evaluation. Journal of Communication Management 7 (4) • McGuire, WJ 1984 Attitudes and Attitude Change in Lindzey, G. & Aronson, E. Ed. The Handbook of Social Psychology Vol.2 3rd Edn, Random House • Walker, G (1997) Public relations practitioners’ use of research, measurement and evaluation, Australian Journal of Communication 24 (2) • Watson T (2005) ROI or evidence-based PR: the language of public relations evaluation. PRism 3, http://praxis.massey.ac.nz/issue_3.html (forthcoming) • Watson T & Noble P (2005) Evaluating PR, Kogan Page, London • Watson T & Simmons P (2004) Public Relations Evaluation – Survey of Australian Practitioners, ANZCA04, July 2004

24. Web sources • www.instituteforpr.com • www.carma.com.au

25. Up and away!

26. Key thinking “PR is concerned with ill-defined problems. It cannot be evaluated by a single method or metric because it does not deal with simple issues …” “Likewise, business in general recognises that a simple, single financial measure does not give an accurate reflection of a company’s true worth.” Gregory & Watson 2006: 1

27. Policy and resources • Research in the UK sponsored by CIPR • Produced by Henley Management College • Looked at practitioner and academic literature.

28. CIPR policy 1 • PR is part of the management task and is subject to the same disciplines, such as the need to set direction, allocate and manage resources, and monitor progress.

29. CIPR policy 2 • Measurement and evaluation are problematic in all areas of management. Complexity is a key factor and in … business, it is difficult to separate out the effect of one area of management such as public relations.

30. CIPR policy 3 • However, the situation can be eased by better planning and objective setting where precise, measurable objectives allow for enlightened judgements of progress in campaigns and their management.

31. CIPR policy 4 • PR can be measured and evaluated in terms of: • Contribution to social and economic development • Contribution to management, leadership and organisational performance by aiding better decision-making and avoiding mistakes • Being a process and part of programme development and implementation • The contribution of individual practitioners

32. CIPR policy 5 • Available methods, research based, provide information that is good enough for decision making for PR programmes • Existing methods … are adequate for measuring the contribution of PR. • Each method has limitations, so a raft of measures appropriate to particular situations needs to be employed.

33. The CIPR resources • Go to www.cipr.co.uk/research

34. Most Admired Companies • A range of evaluation metrics were used – between four and eight. • Most frequent were informal and/or qualitative such as journalist feedback and discussions with stakeholders. • “ ‘Just checking’ everything is on track”. Gregory, Morgan & Kelly 2005