1 / 33

Building foundations for a successful multivariate testing strategy

Building foundations for a successful multivariate testing strategy. Maria Murphy User Experience Manager Michael Schirrmacher UK Sales Director, Webtrends Webtrends | Nov 2012. # WTistrategy. Why should we Optimize?. #1 . Improve Customer Experience

soleil
Download Presentation

Building foundations for a successful multivariate testing strategy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Building foundations for a successful multivariate testing strategy Maria Murphy User Experience Manager Michael Schirrmacher UK Sales Director, Webtrends Webtrends | Nov 2012

  2. #WTistrategy

  3. Why should we Optimize? #1. Improve Customer Experience #2. Improve Conversion & Revenue #3. Deepen Visitor Engagement

  4. Where do I start? Analytics Optimization Identify and quantify opportunities through insight Test, segment and target to improve performance

  5. What Optimisation Involves?

  6. Full Factorial….Fractional factorial….? “ok this looks complex”

  7. So, for now…a managed service is still more popular

  8. Optimization Maturity Model “Walk” Phase “Crawl ” Phase “Run” Phase Operational Reporting

  9. Multivariate Test Control £

  10. Multivariate Test Control Challenger £

  11. +£3.2m Lift in special offers sales in UK Multivariate Test

  12. Web optimization & MVT Realizing the value for your business User Experience Manager Maria MurphY

  13. Introducing Rakuten’s Play.com • Founded 1998 • Acquired October, 2011 • Over 15 million total customers • Over 370,000 Facebook fans • 5 million visits, 33 million page views weekly • Over 27.5 million items listed • Over 3,000 Marketplace Merchants • Over 40% Marketplace growth year over year

  14. What MVT can do MVT can optimize • Design & styles • Page layout • Manipulate display • Inject new content • Redirect URLs • Change journeys • More...

  15. What MVT can’t do MVT can’t replace IT Strategic initiatives Complex queries

  16. Managing MVT Structuring MVT • Nominate 1-2 people to lead MVT • Involve & communicate with IT • Discuss MVT activities within marketing team & design teams • Share roadmap & results with the MVT team Spreading the word • Set up an MVT email address • Share results • Present findings to high-level stakeholders • Run seminars Feed the Optimization Roadmap Optimization Roadmap

  17. Our Approach Engage! • Weekly calls are an important part of the service • Invest time in planning and preparing • Build your own roadmap of ideas • Use your Optimization consultant • Follow-through Commitment • Plan your test • Approve your test • Test your test • Understand your results • Implement Realise the value! Implement & earn!

  18. Test Case 1 Login Page

  19. Login Page 07/2011

  20. Login Page The Sign In page is a key page in the funnel and currently includes some non-vital elements on the page as well as omitting some details which could enhance conversion to next step. It therefore presents a very good testing opportunity.

  21. MVT – Login Page 07/2012

  22. MVT – Login Page Login - optimal Basket Delivery options • Traffic Directly from Basket • +4.32% lift in visitors reaching delivery options page • +5.94% lift in visitors reaching the Create Account page

  23. MVT – Login Page Login - optimal Not from basket Delivery options Check account Check orders Traffic not Directly from Basket: +32.79% lift in visitors reaching the Create Account page Create new customers

  24. Test Case 2 order confirmation

  25. Why MVT

  26. Order Confirmation Page 06/2011

  27. Order Confirmation Page Hypothesis: By making it easier for users to have a final check on their delivery address, card details and basket items and by making the call to action more impactful, conversion to confirmation will increase.

  28. Order Confirmation Page Optimal +2.11% Lift in confirmations Sometimes the simplest things give the biggest results Large “Confirm order’ buttons generated a + 2.11% lift Redesigning the page to use a small order summary box at the top right hand corner had a significantly negative impact on orders

  29. Why MVT

  30. GROWConversionRevenueEngagement

  31. Conclusion Do’s • Build understanding of the capabilities of your optimization tool. • Resource the optimization function • Preach the value of tests and results • Engage with your optimization provider • Plan your optimization programme • Commit time to ensure ROI • Don’t underestimate the value it can add • Don’t under-invest Don’ts

  32. Maria.murphy@play.com MVT Maria MurphY Questions Thank you

More Related