Presentation Transcript

1. Maximize Responses with A/B Testing An Oracle | Eloqua Power Hour Powered by
2. Presenters Isabel Chetta – Oracle, Marketing Advisor - isabel.chetta@oracle.com Molly Thornburg – Oracle, Data Analyst – molly.thornburg@oracle.com
3. Power Hour Agenda Who is doing it? What is A/B testing Definition, how to get started, what to test and what to measure Metrics and benchmarks Defining metrics, building benchmarks and next steps Numbers and examples Why should I do it? What are other companies doing? A/B Testing checklist and additional resources
4. Who is doing it?
5. A/B testing Who is doing it? For 2 years running, A/B testing is the most used method for improving conversion Source: Econsultancy Organizations that very routinely track tests through the entire funnel 64% of B2C organizations 52% of organizations that sell to both business and consumers 56% of organizations with more than 100 employees Source: Marketing Sherpa 2014 Google ran more than 7,000 A/B tests in 2011 alone Source: Pardot Gmail once tested 50 shades of blue for their CTA color and found the highest converting shade Source: Quick Sprout
6. What is A/B testing?
7. What is A/B testing? By definition: AB testing is a strategy used in marketing in which two versions (A and B), a control and a treatment version are tested against each other. The goal is to identify the changes that will increase the chance that what you WANT to occur WILL occur. Source: whispond technologies
8. What is A/B testing Process outline Develop a baseline (benchmark) for testing Determine your testing methodology Determine your hypothesis Execute test Analyze results Optimize based on your test result Repeat the testing process
9. What is A/B testing How to get started: executing the test Pick one element to test Create your testing segment based on your target list Divide your testing into 2 equal groups Send out test Wait +/- 2 business days Collect results Send out winning version to remaining target list
10. Email A Email A What is A/B testing An example: methodology and execution set up 20% Open 5% CTR 20% Inquiries OfferA on Landing Page 25% Open 7% CTR 25% Inquiries OfferB on Landing Page
11. What is A/B testing What am I trying to (dis)prove? Is the messaging easy to understand? Is the call to action more compelling? What are barriers for prospects to convert to a lead? Are we asking for too much information? Do prospects understand the offer and find it valuable? Which topics resonates the best with this segment? Which layout drives the highest conversion? Which images create higher levels of engagement?
12. What is A/B testing What to test? Email Optimal send time (day / time) Lay out Design Subject line From line Call to action Messaging relevance Tone Personalization
13. What is A/B testing What to test? Landing Page Design Lay out Call to action Form completion (conversion) Tone Variations (campaign specific, thank you page, segmented to audience...) Personalization
14. What is A/B testing What to test? Form Design Placement Size Button text Button placement Form completeness Number of fields Tone Personalization
15. Metrics and Benchmarks
16. Metrics and benchmarks Which Metrics to track: Open rate Click through rate Page visits Form conversions Revenue Quality Rate
17. Metrics and benchmarks Benchmarks and delivering results Determine how often to measure results Determine how long to run a test Evaluate multiple cycles Consider A/B testing throughout the funnel Make decisions about campaign strategy, tactics and assets Develop a continuous audit / action approach to A/B testing Only 31% of marketers very, or somewhat routinely, review tests and decide on follow-up tests Source: Marketing Sherpa
18. Sample Email Testing Matrix
19. Numbers and examples
20. Some numbers Benefits of testing? Benchmark studies have shown that the conversion rate of a typical campaign landing page can be increased up to 40%, on average, with a few well designed tests.Source:Whichtestwon.com Around three in five companies surveyed say A/B testing and multivariate testing are ‘highly valuable’ for improving conversion rates. Source: Econsultancy Of companies who carry out website testing, 87% of say they perform between one and five tests each month. Source: Econsultancy Source: Marketing Sherpa 2012
21. Some examples (placeholder slide) What are they testing? Landing page lay-out Long copy vs. short copy Form Elements
22. Which test won? Landing page lay-out
23. Which test won? Version A, the page with a grid of static images, increased clicks by 187.9% and sales by 75.07% at a 99.9% confidence rate
24. Which test won? Long copy vs. short copy (email) Version A Version B
25. Which test won? Version B Click Rate: 0.72% CTOR: 7.84% US Registrations: 33 Total Registrations: 144 Click Rate: 0.61% CTOR: 7.10% US Registrations: 38 Total Registrations: 143
26. Which test won? Form Elements Version A Version B
27. Which test won? Version B 31% more sign-ups Source:
28. Final Check List and Additional Resources
29. A/B Testing check list Let’s get started! Define your goals What are you going to test? Define your testing methodology Set-up and test a control group Collect and analyze the data Identify Trends Implement successful changes Develop continuous testing methodology
30. Resources Read It      Blog: Testing Results - Comic book vs Traditional Email Styling      Blog: Email Deliverability Testing FAQ      Blog: Which Test One - Awards inspire testing      Blog: Will Dynamic Content Change your Engagement Strategy?      Blog: Customer Story: Balluff - Adventures with A/B Testing      Blog: Customer Story: Sonic Foundry Boosts CTR by 224% Learn It Eloqua University: Power Hour A/B Testing Do It      How To: Building a Random A/B Split Email Test Program      How To: How to Run A/B Email Testing on the Campaign Canvas      How To: Improving Email Effectiveness with Text      How To: Math Equations within Marketing Automation (advanced)      How To: A/B Testing Bible
31. Q & A
32. Thank you for your time today! Presenter Contact Details: Isabel.chetta@oracle.com Molly.thornburg@oracle.com