1 / 33

Better Industrial and Scientific Experiments: The Overview and New Directions

Better Industrial and Scientific Experiments: The Overview and New Directions. by James M. Lucas and Derek F. Webb 2002 Fall Technical Conference Valley Forge,PA October 17-18, 2002. James M. Lucas J. M. Lucas and Associates 5120 New Kent Road Wilmington, DE 19808 (302) 368-1214

mackenzie-carrigy
Download Presentation

Better Industrial and Scientific Experiments: The Overview and New Directions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Better Industrial and Scientific Experiments: The Overview and New Directions by James M. Lucas and Derek F. Webb 2002 Fall Technical Conference Valley Forge,PA October 17-18, 2002

  2. James M. Lucas J. M. Lucas and Associates 5120 New Kent Road Wilmington, DE 19808 (302) 368-1214 JamesM.Lucas@worldnet.att.net Derek F. Webb Bemidji State Univ. HS-341, Box 23 1500 Birchmont Dr. NE Bemidji, MN 56601 (218) 755-2846 DWebb@bemidjistate.edu Contact Information J. M. Lucas and Associates

  3. PRELIMINARIES How do you run Experiments?

  4. QUESTIONS • How many of you are involved with running experiments? • How many of you “randomize” to guard against trends or other unexpected events? • If the same level of a factor such as temperature is required on successive runs, how many of you set that factor to a neutral level and reset it? J. M. Lucas and Associates

  5. ADDITIONAL QUESTIONS • How many of you have conducted experiments on the same process on which you have implemented a Quality Control Procedure? • What did you find? J. M. Lucas and Associates

  6. COMPARING THE RESIDUAL STANDARD DEVIATION FROM AN EXPERIMENT WITH THE RESIDUAL STANDARD DEVIATION FROM AN IN-CONTROL PROCESS MY OBSERVATIONS EXPERIMENTAL STANDARD DEVIATION IS LARGER.1.5X TO 3X IS COMMON. J. M. Lucas and Associates

  7. Huey Ju Jeetu Ganju Frank Anbari Malcolm Hazel Derek Webb John Borkowski Research Team J. M. Lucas and Associates

  8. IMPLICATIONS • HOW SHOULD EXPERIMENTS BE CONDUCTED? • “COMPLETE RANDOMIZATION” • (and the completely randomized design) • RANDOM RUN ORDER • (Often Achieved When Complete Randomization is Assumed) • SPLIT PLOT BLOCKING • (Especially When There are Hard-to-Change Factors) J. M. Lucas and Associates

  9. Results for Experiments with Hard-to-Change and Easy-to-Change Factors • One H-T-C or E-T-C Factor: use split-plot blocking • Two H-T-C Factors: may split-plot • Three or more H-T-C Factors: consider RRO or Low Cost Options • Consider “Diccon’s Rule”: Design for the H-T-C Factor J. M. Lucas and Associates

  10. Not Resetting Factors • Common practice • Not addressed by the classical definition • Gives a split-plot blocking structure with the blocks determined at random • May be cost effective • Causes biased hypothesis tests over all randomizations (Ganju and Lucas 1997) J. M. Lucas and Associates

  11. RRO EXPERIMENTS • (Random Run Order Without Resetting Factors) • OFTEN USED BY EXPERIMENTERS • NEVER EXPLICITLY RECOMMENDED • ADVANTAGES • Often achieves successful results • Can be cost-effective • DISADVANTAGES • Often can not be detected after experiment • is conducted (Ganju and Lucas 99) • Biased tests of hypothesis (Ganju and Lucas 97, 02) • Can often be improved upon • Can miss significant control factors J. M. Lucas and Associates

  12. AN ESSENTIAL INGREDIENT OF RANDOM RUN ORDER (RRO) EXPERIMENTS (DuPont QM&TC) “BOLD” EXPERIMENTATION J. M. Lucas and Associates

  13. ADVANTAGES OF COMPLETE RANDOMIZATION • INDEPENDENT ERRORS • GUARDS AGAINST TRENDS AND CYCLES • VALIDATES RANDOMIZATION TESTS • SIMPLE ANALYSIS NOTE: IN ADDITION TO A RANDOM RUN ORDER, THE PHYSICAL ACT OF RESETTING IS NEEDED TO ACHIEVE “COMPLETE” RANDOMIZATION J. M. Lucas and Associates

  14. DISADVANTAGES OF COMPLETE RANDOMIZATION • MORE TIME REQUIRED • MORE EXPENSIVE • LESS EFFICIENT • For easily changed factors J. M. Lucas and Associates

  15. Defining the Properties of Random Run Order Experiments J. M. Lucas and Associates

  16. With one hard-to-change factor, the value of p is • 1, for a completely restricted experiment; • 0, for a completely randomized experiment; • , for a random run order. A Fundamental Theorem Theorem 1. The expected covariance matrix, V, for an experiment, which uses a random run order, is: Ju 1992, Ju and Lucas 2002, extended by Webb 1999, Webb, Lucas and Borkowski 2002 J. M. Lucas and Associates

  17. Some Examples of Super-Efficient Experiments Optimum Blocking with one or two Hard-to-Change Factors

  18. Obs. A B C D Blk 1 - - - - 1 2 - - + + 1 3 - + + - 1 4 - + - + 1 5 + - - - 2 6 + - + + 2 7 + + - + 2 8 + + + - 2 Obs. A B C D Blk 9 - - - + 3 10 - - + - 3 11 - + + + 3 12 - + - - 3 13 + - - + 4 14 + - + - 4 15 + + - - 4 16 + + + + 4 24 with one Hard-to-Change Factor J. M. Lucas and Associates

  19. Obs. A B C D Blk. 1 - - - - 1 2 - - + + 1 3 - + + - 2 4 - + - + 2 5 + - - - 3 6 + - + + 3 7 + + - + 4 8 + + + - 4 Obs. A B C D Blk. 9 - - - + 5 10 - - + - 5 11 - + + + 6 12 - - - - 6 13 + - - + 7 14 + - + - 7 15 + + - - 8 16 + + + + 8 24 with two Hard-to-Change Factors J. M. Lucas and Associates

  20. 24 Split-Plot is Super Efficient • Main Effects plus interaction Model • 11 Terms = (1 + 4 + 6) • For I and A the variance is 02/16 + 12/4 • For other terms it is 02/16 • All terms of a CRD have (02 + 12 )/16 • G-efficiency= 11(02 + 12)/(11 02 + 8 12) J. M. Lucas and Associates

  21. 24 with two Hard-to-Change Factors • Nest Factor B within each A block giving a split-split-plot with 8 Blocks • I=A1=BCD1=ABCD1=B2=AB2=CD2=ACD2 • I and A have variance 02/16 + 12/4 +22 /8 • B, AB and CD have 02/16 + 22 /8 • Other terms have variance 02/16 • G-efficiency = 11(02+12+22)/(1102+812+1022 ) >1.0 J. M. Lucas and Associates

  22. IMPLICATIONS Optimum Block Size (Considering Costs) 2 Blocks Block Size = 8 4 Blocks Block Size = 4 Lambda r = $Hard/$Easy is the Ratio of the Costs of Changing the Hard-to-Change Factor and the Easy-to-Change Factors. LAMBDA is the Ratio of the Hard-to-Change Factor's Variance Component and the Other Variance Component. 24 Design Main Effects and 2 Factor Interactions Model r J. M. Lucas and Associates

  23. 26-1 with one or two Hard-to-Change Factors • Main Effects plus interaction Model • 22 Terms = (1 + 6 + 15) • Use Resolution V, not VI with I=ABCDE Use four blocks I=A=BCF=ABCF=BCDE=ADEF=DEF • Nest Factor B within each A block giving a split-split-plot with 8 Blocks • =B2=AB2=CF2=ACF2=CDE2=ABDEF2=BDEF2 • I and A have variance 02/32 + 12/4 +22 /8 • B, AB and CF have 02/32 + 22 /8 • Other terms have variance 02/32 • G-efficiency = 22(02+12+22)/(2202+1612+2022 ) >1.0 • Drop 22 terms for one h-t-c factor results J. M. Lucas and Associates

  24. Super Efficient Experiments are not always Optimal Example : 24 Main effects model

  25. 24 Main Effects Model • Super-Efficient 8 blocks design with I=A=BC=ABC=CD=ACD=BD=ABD • V(b0) = V(b1) = A2/8 + 2/16 •  V(b2) = V( b3)= V( b4) = 2/16 • max variance = (4A2 + 52)/16 • Design can be improved upon when A2/2 >2.5 by a 12-block design J. M. Lucas and Associates

  26. 12-Block 24 Design • I = CD in four blocks; I = -CD in eight blocks Randomized four block weights V(b0) = V(b1) (A2 + 2)/8 A2/4 + 2/8 2/3, 1/3 V(bi,i>1) (A2 + 2)/8 2/8 0, 1 • The maximum variance is: max variance = A2/6 + 372/72 • Super-Efficient max variance = (4A2 + 52)/16 • A2=3, 2=1 gives 73/72 vs 17/16 J. M. Lucas and Associates

  27. IMPLICATIONS SUPER EFFICIENT EXPERIMENTS (With One or Two Hard-to-Change Factor) SPLIT PLOT BLOCKING GIVES HIGHER PRECISION AND LOWER COSTS THAN COMPLETELY RANDOMIZED EXPERIMENTS J. M. Lucas and Associates

  28. Random Run Order Experiments See Webb, Lucas and Borkowski handout Useful for RSM and for more than two H-t-C Factors

  29. Response Surface Example Good Experimental Practiceand Split-Plot Analysis

  30. One Factor at a Time(OFAT)Experiments J. M. Lucas and Associates

  31. 2k Changing O-F-A-TThe lowest cost 2k experiment J. M. Lucas and Associates

  32. Other O-F-A-T 2k Designs • More Balanced • 2-2-3 Changes instead of 1-2-4 • Least Expensive Way to Run • Require large S/N Ratio J. M. Lucas and Associates

  33. Conclusions • Developed properties of RRO experiments • Given Implications • Exciting research area • Much more to do J. M. Lucas and Associates

More Related