1 / 97

Chapter 4: Design of Experiments

6. Chapter 4: Design of Experiments. 6. Chapter 4: Design of Experiments. Objectives. Explain the role of experiments in answering business questions. You Need to Know. Work is full of questions that you need answers to. Some have answers that only require a lookup:

ayasha
Download Presentation

Chapter 4: Design of Experiments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 6 Chapter 4: Design of Experiments

  2. 6 Chapter 4: Design of Experiments

  3. Objectives • Explain the role of experiments in answering business questions.

  4. You Need to Know • Work is full of questions that you need answers to. • Some have answers that only require a lookup: • What is the policy regarding the use of demographic variables in predictive models? • When did you last send a marketing e-mail to segment 17? • Some do not have readily available answers : • Does it really matter whether you use first-class postage when sending direct mailings for a cruise line? • How should you advertise if you want to maximize sales/expenditure ratio for football tickets?

  5. Statistical Models Can Answer Questions • The models that you learn to use in this course can answer many of the questions that you have. • Do you have the data to perform an analysis and answer the question? • Did you account for the kinds of variables that are in your control as well as the kind of variables over which you have no control?

  6. Questions Often Mean Comparing Things • Does your question imply that a comparison is needed? • First-class versus bulk-rate postage • Primetime versus late-night advertising • Did you conduct an experiment?

  7. Consider This… What is the question that you want to answer? What is the population that you want the answer to pertain to? What kinds of things do you want to compare that you can control? How is the outcome measured (Yobs)? What else impacts Yobs that you cannot control?

  8. Consider This… • What is the question that you want to answer? 1. Does postage make a difference in the response rate? 2. Is it worth the extra expense to advertise tickets for a football game in primetime?

  9. Consider This… What is the question that you want to answer? What is the population that you want the answer to pertain to? The “luxury traveler” segment Football fans

  10. Consider This… What is the question that you want to answer? What is the population that you want the answer to pertain to? What kinds of things do you want to compare that you can control? The class of postage on the offer envelope Whether the tickets are advertised during primetime (expensive) or late night (inexpensive)

  11. Consider This… What is the question that you want to answer? What is the population that you want the answer to pertain to? What kinds of things do you want to compare that you can control? How is the outcome measured (Yobs)? The number of responses from each postage group Ticket sales in the week following each type of advertisement

  12. Consider This… What is the question that you want to answer? What is the population that you want the answer to pertain to? What kinds of things do you want to compare that you can control? How is the outcome measured (Yobs)? What else impacts Yobs that you cannot control? Gender, vacation already taken that year, children Team’s season performance (wins, losses), disposable income of viewing markets, broadcasting lineup

  13. Who Cares about Things You Cannot Control? You do! Only accounting for the things in the experiment that you can control:

  14. Who Cares about Things You Cannot Control? You do! Accounting for the things in the experiment that you can control plus one thing that you cannot control:

  15. Who Cares about Things You Cannot Control? You do! Accounting for the things in the experiment that you can control plus two things that you cannot control:

  16. Consider This… What is the question that you want to answer? What is the population that you want the answer to pertain to? What kinds of things do you want to compare that you can control? How is the outcome measured (Yobs)? What else impacts Yobs that you cannot control? Work smarter: design an experiment!

  17. Idea Exchange • Have you ever conducted an experiment? If so, what was the business or scientific objective? • Web-based experiments are popular because they are relatively inexpensive to implement and they can be modified in real time. Can you describe any Web experiments you have seen? • What kinds of factors might influenceclick-through behavior on, for example, an ad for insurance? For retailclothing? Other types of products and services?

  18. 6 Chapter 4: Design of Experiments

  19. Objectives • Define experimental design concepts and terminology. • Relate experimental design concepts and terminology to business marketing concepts and terminology.

  20. Basic Terms in Design of Experiments (DOE) Response Balance Factor Replication Factor Level Effect Orthogonality Power Treatment Experimental Unit

  21. Basic Terms in DOE: Response A response is the dependent variable of interest in the analyses. It is sometimes called the target or dependent variable. Examples include the following: • Response rate to direct mail solicitations • Default (“Bad”) rate among credit customers • Balance transfer amount • Fraud • Number of items purchased from a catalog • Spend, six months after acquisition

  22. Basic Terms in DOE: Factor • A factor is an independent variable that is a potential source of variation in the response metric. • Examples include the following: • Teaser or introductory APR • Color of envelope • Balance transfer fee • Presence or absence of a sticker on a catalog • First-class versus third-class mail • Others?

  23. Basic Terms in DOE: Factor Level • A factor level is a particular value, or setting, of a factor. • Examples include the following: • 1.99% introductory APR • White envelope • 2% balance transfer fee • Airline mile reward offer • Third-class mail • Others?

  24. Basic Terms in DOE: Effect An effect captures and measures the relationship between changes in factor levels and changes in the response metric.

  25. Examples of an Effect • A offer with a sticker on it garners $10 more, in purchases, than a offer without.

  26. Examples of an Effect • A offer with a sticker on it garners $10 more, in purchases, than a offer without. • The white envelope has a 22% higher response rate than the grey envelope.

  27. A 1% increase in Introductory APR yields a 20% decrease in response rate. • A offer with a sticker on it garners $10 more, in purchases, than a offer without. • The white envelope has a 22% higher response rate than the grey envelope.

  28. Basic Terms in DOE: Treatment • A treatment is a combination of all of the factors, each at one level. In a typical marketing context, a treatment constitutes a unique offer. • Examples include the following: • 1.99% Intro Rate, in a White Envelope, no BT Fee • 0% Intro Rate, in a Grey Envelope, 2% BT Fee • 1.99% Intro Rate, in a Grey Envelope, 2% BT Fee • 0% Intro Rate, in a White Envelope, no BT Fee • There are eight possible treatments when you have three factors, each at two levels.

  29. Basic Terms in DOE: Treatment • A treatment is a combination of all of the factors, each at one level. In a typical marketing context, a treatment constitutes a unique offer. • Examples include the following: • 1.99% Intro Rate, in a White Envelope, no BT Fee • 0% Intro Rate, in a Grey Envelope, 2% BT Fee • 1.99% Intro Rate, in a Grey Envelope, 2% BT Fee • 0% Intro Rate, in a White Envelope, no BT Fee • There are eight possible treatments when you have three factors, each at two levels.

  30. Basic Terms in DOE: Treatment • A treatment is a combination of all of the factors, each at one level. In a typical marketing context, a treatment constitutes a unique offer. • Examples include the following: • 1.99% Intro Rate, in a White Envelope, no BT Fee • 0% Intro Rate, in a Grey Envelope, 2% BT Fee • 1.99% Intro Rate, in a Grey Envelope, 2% BT Fee • 0% Intro Rate, in a White Envelope, no BT Fee • There are eight possible treatments when you have three factors, each at two levels.

  31. Basic Terms in DOE: Treatment • A treatment is a combination of all of the factors, each at one level. In a typical marketing context, a treatment constitutes a unique offer. • Examples include the following: • 1.99% Intro Rate, in a White Envelope, no BT Fee • 0% Intro Rate, in a Grey Envelope, 2% BT Fee • 1.99% Intro Rate, in a Grey Envelope, 2% BT Fee • 0% Intro Rate, in a White Envelope, no BT Fee • There are eight possible treatments when you have three factors, each at two levels.

  32. Other Terms in DOE • An experimental unit is the smallest unit to which a treatmentcan be applied. • Replication occurs when more than one experimental unitreceives the same treatment. • Power is the probability that you will detect an effect, if one exists.

  33. 6 Chapter 4: Design of Experiments

  34. Objectives • Define multifactor experiments. • State the advantages of multifactor experiments versus a sequence of one-factor-at-a-time (OFAT). • Explain how experimental units should be allocated to the treatments. • Define the term interaction. • Analyze a simple multifactor experiment and identify interactions.

  35. Two Factors, Each at Two Levels • Example: Credit card solicitation with an introductory, or teaser, rate • The introductory (Intro) rate is High or Low. • The go-to (Goto) rate is High or Low.

  36. 0% Intro 2.99% 4.99% Goto 7.99% One Factor at a Time Goto Test Intro = ?? Intro Test Goto = ?? ...

  37. 4.99% Goto 7.99% One Factor at a Time Intro Test Hold Goto constant at 4.99% 0% Intro 2.99% Goto Test Hold Intro constant at 0%

  38. 4.99% Goto 7.99% One Factor at a Time "Intro Test" "Control" 0% Intro 2.99% "Goto Test"

  39. 4.99% Goto 7.99% Typical Volumes 50,000 experimental units 50,000 experimental units 0% Intro 2.99% 50,000 experimental units

  40. Efficiency • VP of Marketing • Either a large numerator • or a small denominator • or both! • Experiment Designer • Can you quantify these terms? • Number of items tested • Margin of error • Financial costs • Total sample size ...

  41. Efficiency • VP of Marketing • Either a large numerator • or a small denominator • or both! • Experiment Designer • Can you quantify these terms? • Number of items tested • Total sample size

  42. Efficiency • VP of Marketing • Either a large numerator • or a small denominator • or both! Experiment Designer Can you quantify these terms? • Two terms: Intro effect and Goto effect • 150,000 observations ...

  43. Efficiency?!? VP of Marketing Either a large numerator or a small denominator or both! Experiment Designer Can you quantify these terms? • Two terms: Intro effect and Goto effect • 150,000 observations ...

  44. 4.99% Goto 7.99% Efficiency?!? 0% Intro 2.99%

  45. 0% Intro 2.99% 4.99% Goto 7.99% Efficiency?!? This test uses only two-thirds of the data. This test uses only two-thirds of the data.

  46. 0% Intro 2.99% 0% Intro 2.99% 4.99% 4.99% 4.99% Goto 7.99% 7.99% 7.99% 0% Intro 2.99% 0% Intro 2.99% One Factor at a Time • There are many different ways to arrange the “same” test. • They all assume no interaction between Intro and Goto. • None of these eliminates the potential for bias in the estimates. Goto 4.99% Goto Goto 7.99%

  47. 0% Intro 2.99% 0% Intro 2.99% Goto 4.99% 4.99% 4.99% Goto 7.99% 7.99% 7.99% 4.99% 0% Intro 2.99% 0% Intro 2.99% Goto 7.99% Pick a Treatment Set Goto

  48. Detecting Interactions between Factors Low Goto ResponseRate High Goto Low High Intro Rate

  49. 0% Intro 2.99% Goto 4.99% 7.99% 4.99% 0% Intro 2.99% Goto 7.99% Factorial Arrangement of the Treatments • Permits the testing and estimation of an Intro x Goto interaction term. • Increases the precision of estimates for the same test volumes. • Can use every individual in every test. • Combinations of factor levels provide replication for individual factors.

  50. 0% Intro 2.99% Goto 4.99% 7.99% 4.99% 0% Intro 2.99% Goto 7.99% Efficiency! Reuse Observations • The Intro test uses every observation.

More Related