MARK2038 Data Base Marketing Strategies II

1. MARK2038 Data Base Marketing Strategies II Week 11 Instructor: Santo Ligotti Email: sligotti@gbrownc.on.ca

2. Testing, Metrics, and Post Analysis

3. This week • Testing, metrics, and post analysis • In-class assignment #5 • Structure/content of final test (July 18th, 2006)

4. Learning Objectives: You just learned: • why testing of DBM programs is important; • 4 steps you can take to test DBM programs; • how to analyze the effectiveness of direct response campaigns including response rate, ROI and cost per response.

5. Planning List Budget Offer/call to action Fulfillment Creative format Messages and copy Response device Testing process Response tracking Financial success measures Campaign Management Process Campaign Planning List Compilation Implementation Measurement

6. List compilation Purchase response lists/compiled lists Ensure any last-minute field edits are complete Select list members Forward records to agency/suppliers Flag records for inclusion in CRM system Campaign Management Process Campaign Planning List Compilation Implementation Measurement

7. Campaign Management Process Campaign Planning • Implementation • Campaign is activated • Customer inquiries and orders are acted upon • Information is received from selected media channels • Measurement • Monitor the results of the campaign for effectiveness • Input recommendations to direct marketing planning List Compilation Implementation Measurement

8. Time to Market • Marketing campaigns require an average of 2.5 months to implement.

9. Reducing Time to Market • The longer the campaign lead time, • The less likely the message will be relevant to its audience… • … and the less likely it will be “highly effective.”

10. Getting the right mix, requires internal partnerships • A partnership between Marketing and Analytics will • maximize campaign results • Involve the data analytics team at the beginning of the campaign to establish key business objectives, pre-analysis, targeting and key metrics/tracking • Continually integrate the data analytics team’s tracking and key insights into future campaigns to maximize ROI of all marketing initiatives

11. The Business Challenge • With increasing pressure from shareholders/analysts to continually improve financial results, marketers need to able to illustrate that their campaigns are delivering strong results • In order to ensure marketing dollars are maximized, data analytics needs to become a key partner in the ongoing measurement & tracking of campaigns • A number of marketer’s are still struggling to demonstrate that their campaigns deliver quantifiable results So how do we as marketers achieve this?

12. LISTEN ACTION DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS Data Analytics is key to CRM Process

13. Analyze customer behaviour to determine key drivers of value • Know recent key events and interaction with your company Knowing Your Customer starts with Data Analytics LISTEN ACTION DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

14. Continuously target and tailor offerings based on testing and learning Utilizing Data Analytics allows you to Identify Potential Customer Actions LISTEN ACTION DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

15. Explicitly manage the flow and sequence of marketing communications to each customer Marketing and Data Analytics allows you to Create Appropriate Message LISTEN ACTION DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

16. Create a dynamic and • consistent messaging and • response capability at all • customer touch • (communication) points Marketing Delivers the Message to the Customer LISTEN ACTION DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

17. Capture and remember relevant customer conversations Data Analytics allows you to Listen to the customers response LISTEN ACTION DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

18. Data Analytics allows you to Track the Customer Responses and gain Insights LISTEN ACTION • Customer responds to the message • Key Learning’s are integrated into future programs by marketing DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

19. Establishing a Test & Learn Partnership between marketing & data analytics will maximize results LISTEN ACTION • Conduct sophisticated tests, share learning widely, and implement fast read and re-launch capability DELIVER MESSAGE TEST AND LEARN KNOW THE CUSTOMER CREATE APPROPRIATE MESSAGE IDENTIFY POTENTIAL CUSTOMER ACTIONS

20. The Concept of Testing Why Test? • Good economics: • Use a sample to learn what works and what doesn’t work before rolling to entire database • Continuous Improvement • Learn how to improve marketing programs to ensure they’re the most effective

21. Testing Multiple Variables • Test all or some variables • Why? • Learning Loop: Generates constant feedback on how to improve effectiveness of communications • Commonly tested variables: • Lists • Offers • Creative execution • Channel • Content

22. Testing an Idea Four Steps • Plan Test • Define objectives • Set up test and control groups • Execute Test • Track Results • Analyze Results • Response rate • ROI • Cost per response • LTV

23. Example - Department Store Assumptions • Store has a house credit card tied to customer database containing 400,000 men and women • Store credit card allows capture of information about purchases • Store has new line of designer clothes for women, being promoted through print ads • Would like to increase sales of new clothing line • Decide to test a direct mail program with a small group of women customers, before roll out to entire database • Offer: If buy new suit by May 30, will receive a free piece of costume jewelry worth $20 by presenting this offer

24. Step 1: Plan Test i) Define Marketing Objectives • What are you trying to accomplish? • Objectives should be measurable and time-bound. • Department Store Example: • To increase sales to existing customers by 4% within 1 year. • To achieve sales of new clothing line of $4.2 million. • To increase LTV per customer from $80 to $125 over next 12 months.

25. Total Customers Test Group Control Group Gets Offer Does NOT get offer Step 1: Plan Test ii) Set up test and control groups

26. Why use a Control Group? • Allows you to measure the effect of the promotion versus not running it • No offer or promotional piece sent to the control group • Can be larger/smaller than test group

27. Step 1: Plan Test Set up test and control groups • Query the database to determine how many women have credit cards in their name Example - Department store • 200,000 women with department store credit card in their name • Must select 2 groups from this 200,000: • Women who get the direct mail offer (Test) • Women who do not get the DM offer (Control)

28. Test & Control Groups: How large? • Cost considerations: make as small as possible • Statistical validity: make as large as possible Rule of Thumb: • Each group must be big enough so that you receive at least 500 responses from the promoted group Example • If anticipate response rate of 2% • Test group needs to be (500/2%) = 25,000

29. Test & Control Groups: How large? Example: Department Store • Anticipate response rate of 2.5% • 200K women in database • Test group size = 500/.025 = 20,000

30. Step 1: Plan Test Set up test and control groups • Construct Test Group using ‘Nth’ method (per RFM) • YOUR CONTROL WOULD BE THE SAME FOR THE ENTIRE MAILING UNIVERSE, REGARDLESS OF HOW MANY CELLS Nth = Total customers in database Test Group Quantity • Example: Department store • Test group of 20K • Add another 20K for control group … total = 40K • Nth = 200,000/40,000 = 5 • Select every 5th customer from master database • That is, select customer record #5, #10, #15 ...

31. Why use ‘Nth’ select? • Test and Control groups must be exact statistical replicas of the master database • Must mirror the master database - will have the same percentage of people with similar characteristics: • Same postal code • Same income • Same # of children • Same lifestyle • Same purchase behaviour etc.

32. 200k Women customers Test Group Control Group No Mailed Offer Step 2: Execute Test Execute Program among test group, interacting normally with control group

33. Step 3: Track Results Assign a source code • A “source code” is assigned to each test variable to facilitate measurement and analysis • A source code is a series of letters or numbers used to identify a particular offer • Rule: different source code for each new variable • Example: • Women who got offer: OFFERMAY03 • Women who did not get offer: NOOFFERMAY03

34. Step 4: Analyze Results What is the key learning? 3,000 responses 2,000 responses

35. What is a response? A response can be ... • Phoning a 1-800 number • Providing information (e.g. survey answers) • Entering a contest • Purchasing a product • Signing up for a service Our example

36. Step 4: Analyze Results Evaluate success using a number of factors: • How did the program perform relative to objectives? • Did the promotion come in on budget? Metrics used to analyze performance: • Response Rates Analysis (RR%) • Cost per Response (CPR) • Return on Investment (ROI) • LTV

37. Calculate response rate for Test Group Step 1 Step 2 Calculate response rate for control group Calculate incremental lift between test and control groups Step 3 Response Rate Analysis

38. First, calculate response rate for Test group Response Rate Analysis Department Store Example • Direct mail offer: Get free piece of costume jewelry if buy suit by May 30 • 20,000 mailed, 3,000 responded Test RR% = Responder Quantity x 100=15% Test Quantity

39. Then calculate response rate for the Control group Response Rate Analysis Department Store Example • 20,000 in Control Group do not receive direct mail offer • Still, 2,000 people respond to print advertising and buy a suit by May 30 Control RR% = Responder Quantity x 100 Control Quantity

40. Response Rate Analysis Third, calculate % Lift between groups % Lift = Test RR% – Control RR% x 100 Control RR% Evaluation • The higher the lift, the better • Positive % Lift = Test performed better than Control • Negative % Lift = Control performed better than Test Based on the Department Store example, what is the incremental lift percentage?

41. Cost per Response Analysis Campaign Costs / Budget Include: • Planning & Campaign Development • Agency Costs (e.g. Fees, Creative Development) • List Development (e.g. data work) • Campaign Execution • Printing, Laser/Lettershop, Postage • Response Costs • The marketing cost associated with response to a database marketing campaign • BRC postage, data entry, offer fulfillment, call centre

42. Cost per Response Cost per response = Total cost of program # responses Department Store Example • Total program costs = $210,000 (includes campaign development, execution, response costs) • Cost/response = $210,000/3,000 = $70 Evaluation: the lower the cost, the better

43. Return on Investment (ROI) Analysis • ROI = what you earn on a campaign relative to what you spent on a campaign • Evaluation: the higher, the better • Objective: To determine if you made money from your database marketing investment

44. Return on Investment Analysis ROI = Revenue – Program Costs x 100 Program Costs Department Store Example • Total program costs = $210,000 • Sales revenue = $450/suit=(450*3000) • 3,000 responses to program What is the ROI ?

45. Lifetime Value Next step: • Determine promotion effect on lifetime value • Increased lifetime value, rather than immediate short-term payout, should be the real goal of database marketing • Test effectiveness of alternative ways of increasing LTV

46. Testing an Idea Four Steps • Plan Test • Define objectives • Set up test and control groups • Execute Test • Track Results • Analyze Results • Response rate • ROI • Cost per response • LTV

47. Metrics Example:CIBC: Direct Mail Creative Execution Test

48. Example: CIBC Creative Test • 3 different Direct Mail pieces created for launch of CIBC Adventura Gold Visa card • Packages all the same except the outer envelope: • Cell A: High-end envelope & CIBC logo • Cell B: High-end envelope & Adventura logo • Cell C: High-end envelope & CIBC logo & Aventura logo

49. Example: CIBC Creative Test • Calculate the % lift, cost per response and ROI for each cell • Which envelope creative would you roll out to the entire database of customers?

50. Example: CIBC Creative Test 10,000 3,500