1 / 10

Email Test Case Study

Email Test Case Study. Email Test Scenario. Chase Bank’s email acquisition program with AOL was delivering CPAs consistently well above target. We structured a testing program to immediately introduce new email creative designs, measure response, and optimize email designs accordingly.

vladimir-davenport
Download Presentation

Email Test Case Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Email Test Case Study

  2. Email Test Scenario • Chase Bank’s email acquisition program with AOL was delivering CPAs consistently well above target. • We structured a testing program to immediately introduce new email creative designs, measure response, and optimize email designs accordingly. • The program, thus far, has included 3 individual, sequential tests: • #1 - An all-new email creative execution • #2 - A single variable email iteration • #3 - Subject line testing • Testing was scheduled for May – July, 2007, so learnings could be leveraged in the peak back-to-school & holiday seasons.

  3. Email Test Structure #1 • These 3 email versions ran head-to-head Control Email All New Creative Single Variable Test

  4. Email Test Success #1 • Both the all-new creative & the single variable iteration decreased CPA. Control Email All New Creative Single Variable Test • 12% decrease in CPA • 12% decrease in CPA

  5. Email Test Success #1 • CPAs from both test emails were equally improved, indicating that the new email design is strong, and repositioning the calls-to-action does not adversely affect response. All New Creative Single Variable Test • 12% decrease in CPA • 12% decrease in CPA

  6. Email Test Structure #2 • Next, a multi-variable test with 2 additional CTA buttons was developed. • The new control and the test version ran head-to-head in June. New Control Email Single Variable Test

  7. Email Test Success #2 • This simple multi-variable test drove an incremental 6% decrease in CPA. New Control Email Single Variable Test • 6% decrease in CPA

  8. Email Test Success #3 • Then we tested subject lines head-to-head for each email version: • Subject Line 1: “Special Offer for AOL Members” • Subject Line2 : “Special Rewards for AOL Members” • Subject Line 1, with a single word difference from Subject Line 2, delivered a 15% lower CPA. Decreased CPA by 15% “Special Offer for AOL Members”

  9. Email Test Insights & Actions • The all-new email design drove double-digit decreases in CPA. • Iterations of the all-new design, with repositioned calls-to-action, performed as well as the all-new design in reducing CPA. • Subtle changes to the new control, with additional calls-to-action, decreased CPA still further. • And a single-word difference in subject lines drove additional performance gains. • Continual testing – of both all-new designs as well as small changes to existing designs – is an extremely efficient and highly effective way to significantly improve campaign performance.

  10. Contact Tangible Impact Tangible Impact, Inc. beth@tangibleimpact.com Office: 614.792.3392 Mobile: 614.296.4327

More Related