Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC Professor of Public Communication

1. Measurement and Evaluation Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC Professor of Public Communication University of Technology Sydney Where are we? Where to next?

2. This discussion • A quick update on international standards • Continuing gaps and inconsistencies • Some further barriers to overcome

3. Barcelona Declaration 2010 • Goal setting and measurementare fundamental aspects of any PR program • Measuring outcomesis preferred to measuring outputs • The effect on business [organisation] results should be measured where possible • Media measurement requires quantity and quality • Advertising value equivalents (AVEs) are not the value of PR • Social media can and should be measured • Transparency and replicability are paramount

4. International standards push • Coalition for Public Relations Research Standards • Established 2011 by AMEC, IPR and CPRF • Social Media Standards Conclave (#SMMStandards) • Established 2012 by Coalition + 8 other PR/communication organizations • Global Alliance, PRSA, CIPR, IABC, SNCR, FIBEP, WOMMA, DAA • Working with eight companies • Dell, Ford, General Motors, McDonalds, Proctor & Gamble, Thompson Reuters, South-West Airlines, SAS • Also consulting with • Media Ratings Council, AAAA, ANA, Web Analytics Association • Measurement Summit discussions2011–2013 • Academic input?

6. M&E outputs and guides • Proposed Interim Standards for Metrics in Traditional Media Analysis • Social Media Standard Definitions for Reach and Impressions • Sources and Methods Transparency Table Institute for Public Relations (IPR) Association for Measurement and Evaluation of Communication (AMEC)

7. PRIA Evaluation Model Organisation/Business Objectives Evaluated against objectives Communication Objectives Formative Research PR Activity Undertaken PR Outputs Analysed PR Outcomes Analysed Organisation/ Business Results For example: • Media relations • Publications sent • Events held • Sponsorships undertaken • Communication engaged in • Audience reach • Tone/sentiment of communication • Key messages • Share of voice • Competitor bench- marking • Awareness • Understanding • Consideration • Attitudes/perceptions • Action/behaviour • Relationships • Reputation • Brand positioning • Sales/profits • Market share • Share price • Donations/grants • Customer/staff retention Measurement Evaluative Research Monitoring Reporting Did organisation/business results shift as a consequence? Did activities generate outputs that reached publics effectively and efficiently? What outtakes and outcomes were achieved among key publics and to what extent? PR Outtakes Analysed • Message recall/ retention • Consideration • Responses • Feedback • Engagement Evaluative Research Q:

8. How far we’ve come …

9. Definitions – e.g., ‘reach’ and ‘impressions’

10. Return on Investment (ROI) • PR ROI • Return on Impressions (ROI) • Return on Media Investment (ROMI) • Return on Target Influence (ROTI) • Return on Earned Media (ROEM) • Return on Expectations (ROE) • Return on Communication Investment (ROCI) • Return on Brand Investment (ROBI) • Social media ROI (SROI)

11. Metrics

12. PRIA Evaluation Model Organisation/Business Objectives Evaluated against objectives Communication Objectives Formative Research PR Activity Undertaken PR Outputs Analysed PR Outcomes Analysed Organisation/ Business Results For example: • Media relations • Publications sent • Events held • Sponsorships undertaken • Communication engaged in • Audience reach • Tone/sentiment of communication • Key messages • Share of voice • Competitor bench- marking • Awareness • Understanding • Consideration • Attitudes/perceptions • Action/behaviour • Relationships • Reputation • Brand positioning • Sales/profits • Market share • Share price • Donations/grants • Customer/staff retention Measurement Evaluative Research Monitoring Reporting Did organisation/business results shift as a consequence? Did activities generate outputs that reached publics effectively and efficiently? What outtakes and outcomes were achieved among key publics and to what extent? PR Outtakes Analysed • Message recall/ retention • Consideration • Responses • Feedback • Engagement Evaluative Research Q:

13. The so-called barriers • Cost • Time • Too hard • Management don’t want it

14. Major barriers to address • Conflation of measurement and evaluation • Measurement is taking measures – metrics • Evaluation is determining VALUE • Value = a perceptionfrom a perspective Findings primarily used for reporting Measurement Evaluation Data collection Data analysis Early findings used to fine-tune campaign

15. Major barriers to address • Most measurement and evaluation look backwards • “Tells us what we know intuitively” • “What’s done is done … can’t change the past” • “Post-rationalisation and self-justification” • Look forwards • What insights can improve PR/communication/relationships in future? • What insights can contribute to the organisation’s strategy and outcomes? • Gaps • Trends • Opportunities • Competitor strategy • Emergent issues

16. Obsession with numbers

17. Conclusions • Recognise measurement and evaluation as different and separate • One is about metrics and analytics, the other is about VALUE • Collect qualitativeresearch data as well as quantitative • Noting that many outtakes, outcomes and outflows are perceptions and attitudes (e.g., satisfaction, trust, reputation) and relationships • Look forward, not simply reporting retrospectively • Contribute to organisational strategy and outcomes

18. Questions

19. References/resources • Interim Standards for Metrics in Traditional Media Analysis http://www.instituteforpr.org/topics/proposed-interim-standards-for-metrics-in-traditional-media-analysis • Social Media Standards ‘Sources and Methods Transparency Table’ http://www.smmstandards.com/category/content-sourcing-methods/ • ‘Social Media Standard Definitions for Reach and Impressions’, http://www.smmstandards.com/2013/03/proposed-social-media-standard-definitions-for-reach-and-impressions-from-the-digital-analytics-association/. Also at http://www.smmstandards.com/wp-content/uploads/2013/03/SMM-Standard-Definitions_DAA_v4.pdf • Jim Macnamara, (2014), ‘The development of international standards for measurement and evaluation of public relations and corporate communication: A review’, available at http://www.uts.edu.au/research-and-teaching/our-research/australian-centre-public-communication/research/research-reports • Tom Watson & AnzgarZerfass, (2011), ‘Return on investment in public relations: A critique of concepts used by practitioners from communication and management sciences perspectives’, PRism, vol. 8, no. 1, available at http://www.prismjournal.org/vol8_1.html