Monte Carlo – A Car, A Place, A . . . ?

1. s 6 -6 -4 -2 0 2 4 6 Monte Carlo – A Car, A Place, A . . . ? Six Sigma Alliance, LLC

2. Interactive Presentation Topics • How Is Monte Carlo Simulation Used To Solve Business Problems? • What’re The Steps In A Good Simulation Process • How Does Crystal Ball Software Make the Above “Easy”

3. L1 L2 L3 L4 Predicting Variability How (Today) Can You Predict the Variation in the Gap Dimension? Gap Specifications for Assembly Gap: - 0.0000, + 0.0100 Next Question: Why Would We Care About the Variation in the “GAP?”

4. Some “Variation” Challenges • Design – Can We Produce This “Thing?” • Design - Will This Design Meet Customer Requirements (i.e. Upper/Lower Specifications)? • Finance – What Are the Uncertainties in Our Financial Predictions? • Business Process Design – What Queues Will Build Up, Wait Times for Customers, Average and Variation in Service Time, Cost? • Improvement Projects – If We Buy This Lower (Higher) Cost Part, What Impact Will It Have on Our Quality?

5. Simulation Models • What is a Simulation Model? • A representation of a real system (e.g. computer, mockup, etc.) • What are Simulation Models used for? • Evaluate the trade-offs between performance and resource requirements and to determine the “optimal” design • When are Simulation Models used? • When the process or system is complex • When the risk of failure of the real system is high

6. X1 X2 ? Y Y = f(X1, X2, X3) X3 Monte Carlo Simulation • “Special” Type of Discrete Event Simulation: • The Effect (Y) Can be Described as a Function of the Factors (X’s) • Some or All of the Causes/Factors Can be Described Via Probability Distributions (e.g. Normal, Weibull, Poisson) • Monte Carlo’s Goal: • Describe the Distribution of the Effect (Y) • Monte Carlo Method: • Repeatedly Sample the Input Factors (X’s) and Compute/Record the Effect (Y)

7. Crystal Ball – Monte Carlo Simulator • Features, Functions & Limitations: • Add-on to Microsoft Excel • Can Propagate Variability In X’s To Variability In Y’s • Includes A Wide Variety Of Probability Distributions • Requires That An Explicit Mathematical Relationship Be Defined (Y ~ F[X’s]) • Treats Each Simulation Run As An Independent Event; Doesn’t (Easily) Model Queuing Or Other Throughput Factors • (With OptQuest) Will Optimize The Y as a Function of The X’s

8. L1 L2 L3 L4 Li LTOTAL Typical Crystal Ball Applications • Predict Variability In Overall Component Length As A Function Of Parts’ Variability: • Predict Variability In Revenues, Profit, ROI As A Function Of Variability In Sales, Market Conditions, Expenses, etc. • Predict Reliability Of A Mechanical Device As A Function Of Variability In Stresses Imposed On The Device And The Strength Of Device’s Materials • In General, Predict Probability Of A Desired Outcome As A Function Of Variability In Inputs (i.e., Desired Profit, Sales, Volumes, Risk)

9. Modeling/Simulation Process • Understand Problem To Be Studied And Objective Of Doing Simulation. Develop A Project Plan And Define Roles & Responsibilities • Describe Model Based On Expert Interviews And Observation Of Process • Collect Data Needed To Define Process Properties • Prepare Software Model (Using Crystal Ball, Other Method) • Determine That Computer Model Executes Properly; Compare Model Output With Real Process (If Exists) • Establish The Experimental Options (Scenarios) To Be Simulated • Execute Options (Scenarios) And Collect Performance Measures • Analyze Simulation Results • Make Recommendations 1. Specify 2. Develop 3. Quantify 4. Implement 5. Verify & Validate 6. Plan 7. Conduct 8. Analyze 9. Recommend

10. 1. Specify the Problem • Clarify The Problem/Question • Have Clearly Stated And Accepted Objectives • Get Input From Users, Customers And Stakeholders • Make Certain All Agendas Are Understood • The Customers, Process Owners, Staff, As Well As Management • Clarify Roles And Responsibilities • Customer - Define And Refine Study Goals, Take Action Based On Study Results • Process SMEs* - Provide Information On Product/Process Workings, Support Data Collection • Analyst - Build Accurate Model Of Product/Process To Support Study Goals • Statistician - Perform Input And Output Data Analysis And Evaluation • Develop A Project Plan (e.g. Gantt Chart) Of Modeling/Simulation Project Activities (Integrate With Design or Improvement Project Plan)

11. L1 L2 L3 L4 Crystal Ball Problem – Assembly Gap A Mechanical Assembly Consists of Four Parts Fitted Together as Shown Below: Problem Statement: Given the Current Design Dimensions and Manufacturing Tolerances, What is the Likelihood of an Assembly Being Produced with Excessive Interference?

12. 2. Develop the Model • Define Output Variables (Y’s) Needed To Support Study Objectives • e.g. Physical Quantity, Cycle Times, DPMO • Study Real-World Product or Process • e.g. Interviews, Documentation, Observations • Identify Real-World Counterparts To Model Elements (Xs) • Process: Entities, Activities, Resources • Develop The Mathematical or Logical Relationship Of Model Elements To Output Variables (i.e. Transfer Function) • Engineering, Financial or Physical Relationship • Flowchart/Process Map With Resources • Review Model Against Real-World Process • The Initial “Sanity Check” (The Beginning of Step 5. Verification)

13. L1 L2 L3 L4 The Assembly Gap Model • In This Case, the “Model” is Simple: Assembly Gap = L4 – (L1 + L2 + L3) • Other Models May Include Complex Engineering Relationships, Financial Calculations • Some Models May Include a “Time-Dimension” – e.g. Forecasting Sales, Revenue, Population, Pollution or Radiation Dispersion

14. Seven Principles of Effective Modelers • Stays Oriented Toward Project Goals/Purpose – Model Construction Is Not The End • Includes Only Necessary Detail – Practices “KISS” • Evolves Model Over Time – Starts Simple And Adds Complexity Until The Model Suits The Goals/Purpose • Describes All Critical Activities/Events With Appropriate Detail • Flexible – Makes Model Design Easy To Modify Because It Will Change • Robust – Doesn’t Make Model Applicability Narrow Through Structure Or Assumptions • Clearly Displays Results – Makes Sure All Measured Responses Are Available And Understood

15. Installing Crystal Ball Demo Package • We’ll Lead You Through Installation of the Demo (Good for Seven Days)

16. Crystal Ball Toolbar in Excel Commands to Define Input and Output Variables, Probability Distributions Assumptions include the probability distribution for the variable, plus the necessary parameters to specify the distribution (e.g. mean and standard deviation for the Normal Distribution) Defining a cell as a Forecast tells Crystal Ball that this variable is an output or “Y” for the simulation. Multiple Forecast cells may be defined.

17. Crystal Ball Toolbar in Excel (continued) Commands to Set Up and Run the Simulation, Reset for Another Run and Debug the Simulation (Single Step)

18. Crystal Ball Toolbar in Excel (continued) Commands to View and Analyze the Results of the Simulation Runs; Obtain Crystal Ball Help

19. Crystal Ball Commands Alternate Access to Crystal Ball Commands

20. 3. Quantify the Model “Variable” Variables – Which Factors/Variables Should be Assigned Probability Distributions? Distributions – Which Distributions Best Fit the Variation Associated With the “Variable” Variables? X1 ? Y Y = f(X1, X2, X3) X2 X3 Constants – Which Variables Can Be Left as Constants? (Not Everything Needs to be “Variable”)

21. Available Distributions in Crystal Ball

22. Typical Distributions & Applications * One of Waloddi Weibull’s Original Applications of this Important Distribution

23. Obtaining Model Data • Determine Data Needs From Objectives And Model Elements • Desired Output Variables And Questions About Inputs • Start Data Collection Early • Data Collection Is Time Consuming • Check Existing Databases For Availability • Perform Experiments/Measurements On Real System

24. Fitting Data to a Distribution – Crystal Ball 2. Use Historical Data as Input to Crystal Ball – “Best Fit” Distribution Will Be Recommended . . 1. Choose Fit... Command in Define Assumption Dialog Box . . .

25. 4. Implement the Model • Build The Model In Crystal Ball or Other Simulation Software (ProcessModel, ARENA, etc.) • Define Assumptions (e.g. Probability Distributions) for Input Variables • Define the Output Variables As Forecast Cells • Setup the Simulation Preferences • Test and Debug the Model

26. Implementing the Model The Functional Relationship between the Effect (Y = Assembly Gap) and Factors (X = Part Dimensions) is Entered In Excel:

27. Quantifying the Model Probability Distributions are Assigned to the Factors The Effect is Then Assigned to be a Forecast Cell

28. 5. Verify and Validate Model • Verification Ensuring That The Conceptual Model Is Faithfully Represented By The Implementation (In Software) • ValidationEnsuring That The Model Behaves The Same As The Real System • Apply To Implemented Model And System Outputs • Focuses On Accuracy Of Measured Properties Develop & Quantify Model Implement Model Real Process Verify Validate

29. Running the Model The Distribution of the Effect (Assembly Gap) is Developed Through Repeated Calculations, Drawing Each Calc From the Factors’ Probability Distributions

30. Crystal Ball Output Information/Statistics

31. Interpreting the Model – Output Variation The Likelihood of the Assembly Gap Being Less Than 0” (Interference) Can Be Easily Determined from the Forecast Graph

32. Interpreting the Model – Output Distribution 1. Use Run: Extract Data Command to Obtain “Raw” Output Data 2.Select a Cell, Enter a “Bogus” Value and Click on Define Assumption Button 3.Click on Fit Button and Define Range of Data Corresponding to Output 4.Examine Possible Distributions and Select Best Fit

33. Interpreting the Model - Sensitivity Sensitivity Chart Displays Factors’ Contribution to the “Y” Variation – Identifies Best Opportunities for Improvement

34. 6. Plan Experiments

35. 7. Conduct Experiments What Changes Could Be “Tested” in the Crystal Ball Model to Improve the Gap Performance? 1. 2. 3. 4. 5.

36. 8. Analyze Results • Goals • Use Simulation Data Model To Make Inferences About The Performance Of A Real Or Proposed System • Use Simulation Data Model To Compare The Performance Of Alternative Systems • Method • Determine Point And Interval Estimates For One Or More System Parameters Using Simulation Output • For Alternatives, Perform Hypothesis Testing (Or Other Appropriate Procedure) To Detect Differences

37. The Simulation Analysis Process Real Process Model Simulation Runs Output Analysis SpecLimit • Inferences: • Mean • Variation • Proportions • Sigma, DPMO

38. 9. Make Recommendations • Report Findings And Recommendations To Study Customer(s), Stakeholders • Document Analysis, Assumptions, Models, etc. • Avoid Statistical/Simulation Jargon With Non-Technical People

39. Post Study Actions • When Changes Are Made To Real-World Product or Process, Collect Variable (Xs) And Performance Data (Ys) • Questions: • Does Change/New Process Perform As Predicted? • Are “Gaps” Within Reason, Or As Expected? • In Either Case, Ask “Why?” • Record Lessons Learned For Future Modeling/ Simulation Projects

40. When Not to Use Simulation • When Analytical Models Are Available • Think First, It Can Save Time And Effort • When Behavior Is Deterministic And The System Is Simple • When There Is No Expertise In Output Analysis • When The System Has Intelligent Agents Deciding Actions • In Difficult-To-Model Negotiation or Adversarial Situations

41. Simulation Limitations • Cannot Give Accurate Results If The Data Are Inaccurate (Garbage In, Garbage Out) • Cannot Describe Process Characteristics That Have Not Been Explicitly Modeled • Simulation Does Not Optimize; However It Can Provide The Function To Be Optimized System Response Optimization Search Strategy Simulation Model Control Variables

42. Time for One More??? • Filling Operation: • Average Volume: 1250 cm3 • Std. Deviation: 5 cm3 • Distribution: Normal • Container Molding Process: • Average Radius: 3.99 cm. • Std. Deviation: 0.01 cm. • Distribution: Normal • Key Relationship: h = V/( x r2) • Hint: Excel’s  function is PI() Shampoo Bottle Design Fill Height: 25 cm +/- 0.5 cm

43. References • Tolerance Design – A Handbook for Developing Optimal Specifications, C. M. Creveling, Addison-Wesely, ISBN 0-201-63473-2, 1997. • Simulation Modeling and Analysis, Averill M. Law & W. David Kelton, McGraw-Hill, ISBN 0-07-036698-5, 1991. • Crystal Ball Software, Decisioneering, Inc. (www.decisioneering.com)