Email Test Case Study
This presentation is the property of its rightful owner.
Sponsored Links
1 / 10

Email Test Case Study PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Email Test Case Study. Email Test Scenario. Chase Bank’s email acquisition program with AOL was delivering CPAs consistently well above target. We structured a testing program to immediately introduce new email creative designs, measure response, and optimize email designs accordingly.

Download Presentation

Email Test Case Study

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Email test case study

Email Test Case Study


Email test case study

Email Test Scenario

  • Chase Bank’s email acquisition program with AOL was delivering CPAs consistently well above target.

  • We structured a testing program to immediately introduce new email creative designs, measure response, and optimize email designs accordingly.

  • The program, thus far, has included 3 individual, sequential tests:

    • #1 - An all-new email creative execution

    • #2 - A single variable email iteration

    • #3 - Subject line testing

  • Testing was scheduled for May – July, 2007, so learnings could be leveraged in the peak back-to-school & holiday seasons.


Email test case study

Email Test Structure #1

  • These 3 email versions ran head-to-head

Control Email

All New Creative

Single Variable Test


Email test case study

Email Test Success #1

  • Both the all-new creative & the single variable iteration decreased CPA.

Control Email

All New Creative

Single Variable Test

  • 12% decrease in CPA

  • 12% decrease in CPA


Email test case study

Email Test Success #1

  • CPAs from both test emails were equally improved, indicating that the new email design is strong, and repositioning the calls-to-action does not adversely affect response.

All New Creative

Single Variable Test

  • 12% decrease in CPA

  • 12% decrease in CPA


Email test case study

Email Test Structure #2

  • Next, a multi-variable test with 2 additional CTA buttons was developed.

  • The new control and the test version ran head-to-head in June.

New Control Email

Single Variable Test


Email test case study

Email Test Success #2

  • This simple multi-variable test drove an incremental 6% decrease in CPA.

New Control Email

Single Variable Test

  • 6% decrease in CPA


Email test case study

Email Test Success #3

  • Then we tested subject lines head-to-head for each email version:

  • Subject Line 1: “Special Offer for AOL Members”

  • Subject Line2 : “Special Rewards for AOL Members”

  • Subject Line 1, with a single word difference from Subject Line 2, delivered a 15% lower CPA.

Decreased CPA by 15%

“Special Offer for AOL Members”


Email test case study

Email Test Insights & Actions

  • The all-new email design drove double-digit decreases in CPA.

  • Iterations of the all-new design, with repositioned calls-to-action, performed as well as the all-new design in reducing CPA.

  • Subtle changes to the new control, with additional calls-to-action, decreased CPA still further.

  • And a single-word difference in subject lines drove additional performance gains.

  • Continual testing – of both all-new designs as well as small changes to existing designs – is an extremely efficient and highly effective way to significantly improve campaign performance.


Email test case study

Contact Tangible Impact

Tangible Impact, Inc.

[email protected]

Office: 614.792.3392

Mobile: 614.296.4327


  • Login