No more excuses! How to measure your results when there’s no money for research

1. No more excuses! How to measure your results when there’s no money for research A presentation to IABC Washington DC June 2005 By Katie Delahaye Paine CEO KDPaine & Partners, LLC Publisher The Measurement Standard Member: IPR Measurement Commission

2. Why measure? The World According to Martians • Results = • ROI • Hard Numbers • Charts & Graphs • Work = • Reviewing results • Looking at spreadsheets • Downsizing

3. Work = Schmoozing Lunching Opportunistic creativity The World According to Venutians • Results = • A busy trade show • An award • A front page headline

4. What Martians hear Blah, blah, We got great blah results Let Research be your dictionary What we say

5. Why conduct research? • Gut feeling doesn’t cut it anymore • Accountability is critical • If you think research is expensive, what’s the cost of ignorance? • Without data, how do you know what’s working?

6. The Myths of Research • You can’t measure intangibles • Measurement will show that my program isn't working • Research should be done either at the start or the end of a program • Measurement is expensive

7. How to lose a budget in 10 days • Don’t tie results to business outcomes • Use research to justify your existence • Measure something that no one cares about • Get lost in the minutiae, lose sight of the goal • Fail to get consensus from everyone who needs the results • Promise a Cadillac research plan on a Segway budget • Deliver data when you no longer need the answers

8. 1987 1998 2000 2002 2003 2004 A little research history Delahaye founded Readers From target audience On-line analysis 24/7 Access to data Automated analysis introduced Automated Message tracking Do-it-yourself Tools Integrated Automated tools Surveys @ $20/ complete Avg cost $30 /clip Avg cost $20 /clip Avg cost $10 /clip Avg cost $5/clip Avg cost <$5 /clip

9. Industry standards • Standards and guidelines are available on www.instituteforpr.com • Clip counts and column inches are not research • AVEs are taboo • Analysis of messaging, positioning, issue identification are the norm • Competitive analysis is mandatory • Analyst and quote measurement is the latest trend • Integrating media analysis with web activity, customer outcomes is growing

10. 5 Steps to a Perfect Research Program • Define your success and objectives • Define your needs • Define the specific answers you need • Determine what you are benchmarking against • Select the most appropriate research tool • Analyze results and glean insight • Do it again

11. Step 1: Defining success and objectives • If you are celebrating complete 100% success a year from now, what is different about your organization? • What do you hope to have happen as a result of your research • If you didn’t do any research what would be different?

12. Step 2: Define your needs • The bigger the audience, the more you segment, the more it costs, the more granular your information • New/old • male/female • by location • The more publications/companies/countries you study, the more clips you get, the more money it costs. • Rules of thumb: • Surveys = $15-$50 per completed survey • Content analysis = $6-30 a clip

13. Step 3: Define the specific research: • Attitudinal: Did your target audience see the messages? Did they believe the messages? Did their attitudes towards the product or program change? Did the relationship improve or decline? • Behavioral: Did you get the audience to behave differently? • Media research: Did you get the coverage you wanted? Did it contain the key messages?

14. Typical questions that research can answer: • Do they remember your message • Do they believe your message • Are they going to act on your message • Does the media mention brand benefits? • How is the brand positioned? • Are you getting your fair share of mentions relative to competition? • Do influencers recommend the brand? • What is the context or subject of the mention? • What are the issues discussed? • Are customers more likely to buy? • Are customers expressing interest?

15. Most frequently used criteria • % change in awareness • % change in preference • % change in purchase intent • % increase in prospects or new contacts • Cost per message communicated • CPM • Cost per minute spent with prospect • Strength of relationships • Cost per % of target population reached • Share of recommendations (positive/negative) exposure • Share of visibility • Share of quotes • Share of brand benefits mentioned

16. Past Performance Your competition Peer Organizations Whatever keeps the Martians up at night Step 4: To what do you compare your results

17. Step 5: Selecting a measurement tool

18. Step 5: Deciding what’s best

19. Step 5: Pick a tool: The latest • Clipping services CustomScoop, Cyberalert, e-Watch, Factiva, Dialog, Nexis • Computerized Content Analysis Biz 360, Cymfony • International Analysis Still mostly manual • Automated survey tools Survey Monkey, Zoomerang • Web measurement ClickTrax, Web trends • Integrated Knowledge Management Tools Performa, Vocus

20. Step 6: Analyze and glean insight • Research without insight is just trivia • Figure out what works and what doesn’t work • Determine what you need to do NOW • Determine what you need to do next • Make sure your recommendations are actionable

21. Step 7: Do it again • Regular research is far more valuable than one-shots • Make sure your data is ready when you need it

22. 7 ways to do research without a budget • Become someone’s research project • Involve your board of directors and volunteers • Research something that HAS a budget • Take advantage of free offers • Become a case study • Team up with peer organizations • Analyze data that already exists

23. Case Study: Rensselaer County • Output Measures: • Total opportunities to see key messages • Share of positives vs. negatives • Share of visibility • Share of positioning • Outtake measures • More favorable attitudes • Greater likelihood to approve • Outcome measures • Economic growth • Projects approved

25. Case Study: Rensselaer County • Goal: More favorable attitudes, better image • Benchmark: Saratoga County • Measurement Budget: $7500 • Tools used: IPR Guidelines, phone survey, Excel, Publisher

26. Case Study: Central Arizona Project • Goal: Measure relationships with target audiences – media, elected officials, customers • Survey instrument: IPR Guidelines • Tool: Survey Monkey • Budget: $1000

27. Case study: SPAA • Goal: Grow the association, increase revenues • Survey instrument: UNH Survey research center • Tool: Survey Monkey • Budget: $1500

28. Case Study: Poetry & Politics • List development – December 2002 • Web-site up by January 1, 2003 ($2,500) • First press release distributed: January • Follow up release and email: February • Final release with schedule: April

29. Outputs • 11.5 million opportunities to see information about Poetry and Politics • 8.9 million total opportunities to see “New Hampshire Writers’ Project” • CPM:$.44 • Cost per message communicated: $3.73 • Most frequently quoted: Marie Harris, poet laureate NH • Most frequently communicated message: Poetry is important

30. Calculating exposure and cost per message communicated $11.95 $5.08 $.59 $.46 Numbers indicate cost per message communicated $.56

31. Evaluating the spokespeople

32. Cost of measurement:$600 • Clip monitoring – Custom Scoop and Cyberalert ($0) • Determining OTS – MediaMap, PRTrak, SRDS • COMMAudit : $495 • PowerPoint, Excel

33. Case Study: Southwest Airlines Southwest. COM Vamonos release generated $38,000 in ticket sales Date: February 12, 2004

34. Southwest Airlines made four big announcements on July 15, 2004

35. Release about 22 new daily flights generated $1 million in ticket sales

36. Southwest’s PR has generated over $1.5 million in ticket sales • Service to Philadelphia • Began with 14 daily flights to 6 cities • Expanded to 28 daily flights to 14 cities • Expanding again to 41 daily flights to 17 cities • Revenue passenger miles for the eight months ended August 31, 2004, increased 11% to 36.3 billion Source: Southwest Airlines

37. Thank You! • For more information on measurement, subscribe to The Measurement Standard, www.themeasurementstandard.com • To start developing your own dashboard or for a copy of this presentation go to: http://www.measuresofsuccess.com • Or call me at 1-603-868-1550 • Or give us your business card and we'll be happy to send it to you