1 / 25

Message Testing

Message Testing. MMC5427 Chelsea Bundschuh. What is Message Testing?. Exactly as it sounds! Communications testing by which a baseline control message is evaluated or compared to alternative test messages with the objective of increasing the desired response. It’s an experiment.

winfield-reynold
Download Presentation

Message Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Message Testing MMC5427 Chelsea Bundschuh

  2. What is Message Testing? • Exactly as it sounds! • Communications testing by which a baseline control message is evaluated or compared to alternative test messages with the objective of increasing the desired response. • It’s an experiment.

  3. Why We Don’t Test • We know what the message should be. • We know the target audience. • It costs too much money. • We don’t have the time. http://www.forrestwanderson.com/documents/TestingPRMessages_000.pdf

  4. Why We Should • Improves efficiency • Suggest challenges • Identifies opportunities • Increases likelihood of intended success • Reduces likelihood of unintended reactions • Proves performance

  5. Who Should Test Messages? • Advertisers • Public Relations Practitioners • Politicians • Web Designers • Community Leaders • Graphics Designers • Anyone who is putting out messaging to meet a goal.

  6. “Sophisticated marketers are learning and growing all the time and testing more subtle variables while seeing dramatic gains in results,” Carolyn Goodman — president of Goodman Marketing Partners, a multichannel direct response company in San Rafael, Calif.

  7. Campaign Steps • 1. Define the campaign goal(s) • 2. Research & identify the target audience • 3. Generate messages • 4. TEST & revise messages • 5. Implement • 6. Evaluate http://www.pcs-strategies.com/five-key-steps.html

  8. How To Test Messages • 1. Designate your original message as the control. • 2. Develop message variations or alternatives to the control. • Examples of testable messages: advertising copy, graphics, layouts, tag lines, mission statements, web sites, etc… • 3. Test to measure effectiveness of both control message and variations.

  9. Types of Testing • Qualitative • Focus Groups • In-depth interviews • Projective techniques (word associations, sentence completion, etc…) • Quantitative • Memory tests(day after recall) • Persuasion tests • Direct Response tests (A/B & multivariate) • Continuous measurement tests (dial turning) • Physiological measures ( eye movement, brain-wave analysis) http://www.slideshare.net/jmckeever/advertising-testing-techniques

  10. A/B Testing • Two designs of a website: A and B. • A is the existing design (the control), and B is the new design. • Split your website traffic between two versions. • Measure their performance using metrics or goals.

  11. A/B Testing Do’s & Don’ts • Don’ts • Do not wait to test the variation until after you have tested the control. Always split your traffic. • Do not end the testing too early. Google Analytics recommends at least a two week run to achieve a 95% confidence interval. • Do not surprise regular visitors. • Do not let gut feeling overrule results. • Do’s • Know when to give up if variations are not at all performing. • Make your test consistent across the whole website. • Run LOTS of tests! • http://www.smashingmagazine.com/2010/06/24/the-ultimate-guide-to-a-b-testing/

  12. Google Analytics Content Experiments • Multivariate testing platform. • Launched in 2012, regularly adding new features. • Experiments lets you test which version of a landing page leads to the greatest improvement in goal completion or metric value. • You can test up to five variations of a page. • http://www.google.com/analytics/

  13. http://online-behavior.com/testing/content-experiments

  14. Messages Gone Wrong Ashton Kutcher plays four characters on a fake dating show in billboard and video ads for Popchips snacks. In particular, the character Raj, in brown face, does not go over well with consumers. http://www.youtube.com/watch?v=DLdobzj_9_I Boxed flowers insult a woman in this Teleflora commercial for the 2009 Superbowl. http://www.youtube.com/watch?v=Gf6PKyxm5IY

  15. Bad Political Slogans • Got Guv? • Jim Oberweis’ gubernatorial campaign in Illinois. He lost. • In Your Heart, You Know He’s Right • Barry Goldwater’s presidential campaign against President Lyndon Johnson • It’s the Economy, Stupid! • De facto slogan from President Bill Clinton’s 1992 presidential campaign • I’m not a witch, I’m you! • Political commercial slogan for Christine O’Donnell’s 2010 US Senate seat campaign • Pour It On ‘Em, Harry! • One of President Harry Truman’s slogan for the 1948 presidential campaign

  16. Case Study: Obama for America • Tactic: Message test every day • In 2012, Obama for America tested and sent a national email almost every day, increasing to multiple times a day as Election Day drew nearer. • "Sometimes we’d see a lift of anywhere between 5% or 10%. But 5% or 10% on an email that’s projected to raise a million dollars is a lot of money. It’s totally worth our while, so that’s why we had 20 writers and 20 email staffers working at all hours of the night — to make sure these tests were ready,“ Toby Fallsgraff, Director of Email, Obama for America.http://www.marketingsherpa.com/article/case-study/obama-email-campaign-testing

  17. Obama for America Daily Testing Process • Step 1. Write a bunch of emailsStep 2. Choose four to six and brainstormStep 3. Tailor the copyStep 4. Test the message, then the subject lineStep 5. Start over • Result: The team’s best-performing subject line, "I will be outspent," and best-performing email raised more than $2.6 million. 

  18. What Were They Thinking? Reebok Belvedere

  19. What Were They Thinking? Sources: Piperlime.com Lung Cancer Alliance

  20. What Were They Thinking? ControlTonight.com Nivea

  21. What is Culture? • Geert Hofstede defines it as “the collective programming of the mind distinguishing the members of one group or category of people from another”. The “category” can refer to nations, regions within or across nations, ethnicities, religions, occupations, organizations, or the genders.

  22. Hofstedes’s6 Dimensions Of Culture •  Describes the effects of a society's culture on the values of its members, and how these values relate to behavior. • Six dimensions of national cultures: • 1. Masculinity vs Femininity Index • 2. Long Term vs Short Term Orientation • 3. Power Distance • 4. Uncertainty Avoidance • 5. Individualistic vs Collectivist • 6. Indulgence vs Restraint (NEW 2010) • http://conorneill.com/2012/06/07/geert-hofstede-the-6-dimensions-of-national-culture/

  23. Cultural Differences In Messaging McDonalds television advertisements Japan: http://www.youtube.com/watch?v=_UKLncvGxQ8 Russia: http://www.youtube.com/watch?v=LEwYcYtLkZ4 France: http://www.youtube.com/watch?v=SBuKuA9nHsw

  24. UK vs. French Culture http://www.smartinsights.com/online-brand-strategy/international-marketing/web-design-for-cultural-differences/

  25. What About You? • Have you encountered a communications “what were they thinking?” recently? • Have you or your company ever conducted some sort of message testing before publically disseminating a message? • Have you ever been surprised by what tested positively? • Do you think you are more likely to message test in the future? • Are there times when you would suggest not message testing? What’s realistic?

More Related