Maximize Responses with A/B Testing

Maximize Responses with A/B Testing An Oracle | Eloqua Power Hour Powered by

Presenters Isabel Chetta – Oracle, Marketing Advisor - isabel.chetta@oracle.com Molly Thornburg – Oracle, Data Analyst – molly.thornburg@oracle.com

Power Hour Agenda Who is doing it? What is A/B testing Definition, how to get started, what to test and what to measure Metrics and benchmarks Defining metrics, building benchmarks and next steps Numbers and examples Why should I do it? What are other companies doing? A/B Testing checklist and additional resources

Who is doing it?

A/B testing Who is doing it? For 2 years running, A/B testing is the most used method for improving conversion Source: Econsultancy Organizations that very routinely track tests through the entire funnel 64% of B2C organizations 52% of organizations that sell to both business and consumers 56% of organizations with more than 100 employees Source: Marketing Sherpa 2014 Google ran more than 7,000 A/B tests in 2011 alone Source: Pardot Gmail once tested 50 shades of blue for their CTA color and found the highest converting shade Source: Quick Sprout

What is A/B testing?

What is A/B testing? By definition: AB testing is a strategy used in marketing in which two versions (A and B), a control and a treatment version are tested against each other. The goal is to identify the changes that will increase the chance that what you WANT to occur WILL occur. Source: whispond technologies

What is A/B testing Process outline Develop a baseline (benchmark) for testing Determine your testing methodology Determine your hypothesis Execute test Analyze results Optimize based on your test result Repeat the testing process

What is A/B testing How to get started: executing the test Pick one element to test Create your testing segment based on your target list Divide your testing into 2 equal groups Send out test Wait +/- 2 business days Collect results Send out winning version to remaining target list

Email A Email A What is A/B testing An example: methodology and execution set up 20% Open 5% CTR 20% Inquiries OfferA on Landing Page 25% Open 7% CTR 25% Inquiries OfferB on Landing Page

What is A/B testing What am I trying to (dis)prove? Is the messaging easy to understand? Is the call to action more compelling? What are barriers for prospects to convert to a lead? Are we asking for too much information? Do prospects understand the offer and find it valuable? Which topics resonates the best with this segment? Which layout drives the highest conversion? Which images create higher levels of engagement?

What is A/B testing What to test? Email Optimal send time (day / time) Lay out Design Subject line From line Call to action Messaging relevance Tone Personalization

What is A/B testing What to test? Landing Page Design Lay out Call to action Form completion (conversion) Tone Variations (campaign specific, thank you page, segmented to audience...) Personalization

What is A/B testing What to test? Form Design Placement Size Button text Button placement Form completeness Number of fields Tone Personalization

Metrics and Benchmarks

Metrics and benchmarks Which Metrics to track: Open rate Click through rate Page visits Form conversions Revenue Quality Rate

Metrics and benchmarks Benchmarks and delivering results Determine how often to measure results Determine how long to run a test Evaluate multiple cycles Consider A/B testing throughout the funnel Make decisions about campaign strategy, tactics and assets Develop a continuous audit / action approach to A/B testing Only 31% of marketers very, or somewhat routinely, review tests and decide on follow-up tests Source: Marketing Sherpa

Sample Email Testing Matrix

Numbers and examples

Some numbers Benefits of testing? Benchmark studies have shown that the conversion rate of a typical campaign landing page can be increased up to 40%, on average, with a few well designed tests.Source:Whichtestwon.com Around three in five companies surveyed say A/B testing and multivariate testing are ‘highly valuable’ for improving conversion rates. Source: Econsultancy Of companies who carry out website testing, 87% of say they perform between one and five tests each month. Source: Econsultancy Source: Marketing Sherpa 2012

Some examples (placeholder slide) What are they testing? Landing page lay-out Long copy vs. short copy Form Elements

Which test won? Landing page lay-out

Which test won? Version A, the page with a grid of static images, increased clicks by 187.9% and sales by 75.07% at a 99.9% confidence rate

Which test won? Long copy vs. short copy (email) Version A Version B

Which test won? Version B Click Rate: 0.72% CTOR: 7.84% US Registrations: 33 Total Registrations: 144 Click Rate: 0.61% CTOR: 7.10% US Registrations: 38 Total Registrations: 143

Which test won? Form Elements Version A Version B

Which test won? Version B 31% more sign-ups Source:

Final Check List and Additional Resources

A/B Testing check list Let’s get started! Define your goals What are you going to test? Define your testing methodology Set-up and test a control group Collect and analyze the data Identify Trends Implement successful changes Develop continuous testing methodology

Resources Read It Blog: Testing Results - Comic book vs Traditional Email Styling Blog: Email Deliverability Testing FAQ Blog: Which Test One - Awards inspire testing Blog: Will Dynamic Content Change your Engagement Strategy? Blog: Customer Story: Balluff - Adventures with A/B Testing Blog: Customer Story: Sonic Foundry Boosts CTR by 224% Learn It Eloqua University: Power Hour A/B Testing Do It How To: Building a Random A/B Split Email Test Program How To: How to Run A/B Email Testing on the Campaign Canvas How To: Improving Email Effectiveness with Text How To: Math Equations within Marketing Automation (advanced) How To: A/B Testing Bible

Thank you for your time today! Presenter Contact Details: Isabel.chetta@oracle.com Molly.thornburg@oracle.com

Maximize Responses with A/B Testing