1 / 30

Six Sigma in Measurement Systems: Evaluating the Hidden Factory

OK. First Time Correct. Inputs. Operation. Inspect. NOT OK. Rework. Hidden Factory. Scrap. Time, cost, people. Six Sigma in Measurement Systems: Evaluating the Hidden Factory. Bill Rodebaugh Director, Six Sigma GRACE. Objectives. The Hidden Factory Concept

kane
Download Presentation

Six Sigma in Measurement Systems: Evaluating the Hidden Factory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. OK First Time Correct Inputs Operation Inspect NOTOK Rework Hidden Factory Scrap Time, cost, people Six Sigma in Measurement Systems: Evaluating the Hidden Factory Bill RodebaughDirector, Six Sigma GRACE

  2. Objectives • The Hidden Factory Concept • What is a Hidden Factory? • What is a Measurement System’s Role in the Hidden Factory? • Review Key Measurement System metrics including %GR&R and P/T ratio • Case Study at W. R. GRACE • Measurement Study Set-up and Minitab Analysis • Linkage to Process • Benefits of an Improved Measurement System • How to Improve Measurement Systems in an Organization

  3. OK First Time Correct Inputs Operation Inspect NOTOK Rework Hidden Factory Scrap Time, cost, people The Hidden Factory -- Process/Production • What Comprises the Hidden Factory in a Process/Production Area? • Reprocessed and Scrap materials -- First time out of spec, not reworkable • Over-processed materials -- Run higher than target with higherthan needed utilities or reagents • Over-analyzed materials -- High Capability, but multiple in-processsamples are run, improper SPC leading to over-control

  4. The Hidden Factory -- Measurement Systems OK Production SampleInputs Lab Work Inspect NOTOK Re-test Hidden Factory Waste Time, cost, people • What Comprises the Hidden Factory in a Laboratory Setting? • Incapable Measurement Systems -- purchased, but are unusabledue to high repeatability variation and poor discrimination • Repetitive Analysis -- Test that runs with repeats to improve knownvariation or to unsuccessfully deal with overwhelming sampling issues • Laboratory “Noise” Issues -- Lab Tech to Lab Tech Variation, Shift toShift Variation, Machine to Machine Variation, Lab to Lab Variation

  5. The Hidden Factory Linkage • Production Environments generally rely upon in-process sampling for adjustment • As Processes attain Six Sigma performance they begin to rely less on sampling and more upon leveraging the few influential X variables • The few influential X variables are determined largely through multi-vari studies and Design of Experimentation (DOE) • Good multi-vari and DOE results are based upon acceptable measurement analysis

  6. Objectives • The Hidden Factory Concept • What is a Hidden Factory? • What is a Measurement System’s Role in the Hidden Factory? • Review Key Measurement System metrics including %GR&R and P/T ratio • Case Study at W. R. GRACE • Measurement Study Set-up and Minitab Analysis • Linkage to Process • Benefits of an Improved Measurement System • How to Improve Measurement Systems in an Organization

  7. Observed Process Variation Actual Process Variation Measurement Variation Long-term Short-term Variation Variation due Variation due Process Variation Process Variation w/i sample to gage to operators Repeatability Calibration Stability Linearity Possible Sources of Process Variation We will look at “repeatability” and “reproducibility” as primary contributors to measurement error

  8. LSL USL LSL USL How Does Measurement Error Appear? Actual process variation - No measurement error Observed process variation - With measurement error

  9. Measurement System Terminology • Discrimination - Smallest detectable increment between two measured values • Accuracy related terms • True value - Theoretically correct value • Bias - Difference between the average value of all measurements of a sample and the true value for that sample • Precision related terms • Repeatability - Variability inherent in the measurement system under constant conditions • Reproducibility - Variability among measurements made under different conditions (e.g. different operators, measuring devices, etc.) • Stability - distribution of measurements that remains constant and predictable over time for both the mean and standard deviation • Linearity - A measure of any change in accuracy or precision over the range of instrument capability

  10. Measurement Capability Index - P/T • Precision to Tolerance Ratio • Addresses what percent of the tolerance is taken up by measurement error • Includes both repeatability and reproducibility • Operator x Unit x Trial experiment • Best case: 10% Acceptable: 30% Usually expressed as percent Note: 5.15 standard deviations accounts for 99% of Measurement System (MS) variation. The use of 5.15 is an industry standard.

  11. Measurement Capability Index - % GR&R • Addresses what percent of the Observed Process Variation is taken up by measurement error • %R&R is the best estimate of the effect of measurement systems on the validity of process improvement studies (DOE) • Includes both repeatability and reproducibility • As a target, look for %R&R < 30% Usually expressed as percent

  12. Objectives • The Hidden Factory Concept • What is a Hidden Factory? • What is a Measurement System’s Role in the Hidden Factory? • Review Key Measurement System metrics including %GR&R and P/T ratio • Case Study at W. R. GRACE • Measurement Study Set-up and Minitab Analysis • Linkage to Process • Benefits of an Improved Measurement System • How to Improve Measurement Systems in an Organization

  13. Case Study Background • Internal Raw Material, A1, is necessary for Final Product production • Expensive Raw Material to produce – produced at 4 locations Worldwide • Cost savings can be derived directly from improved product quality, CpKs • Internal specifications indirectly linked to financial targets for production costs are used to calculate CpKs • If CTQ1 of A1 is too low, then more A1 material is added to achieve overall quality – higher quality means less quantity is needed – this is the project objective • High Impact Six Sigma project was chartered to improve an important quality variable, CTQ1 • The measurement of CTQ1 was originally not questioned, but the team decided to study the effectiveness of this measurement • The %GR&R, P/T ratio, and Bias were studied • Each of the Worldwide locations were involved in the study • Initial project improvements have somewhat equalized performance across sites. Small level improvements are masked by the measurement effectiveness of CTQ1

  14. CTQ1 MSA Study Design (Crossed) Site 2 Lab Site 3 Lab Site 1 Lab Site 4 Lab Site 1 Sample 2 Site 2 Sample 1….. Site 1 Sample 1 6 analyses/site/sample 2 samples taken from each site 2*4 Samples should be representative Each site analyzes other site’s sample. Each plant does 48 analyses 6*8*4=196 analyses Op 1 Op 2 Op 3 T2 T1

  15. CTQ1 MSA Study Results (Minitab Output)

  16. CTQ1 MSA Study Results (Minitab Session) Source DF SS MS F P Sample 7 14221 2031.62 5.0079 0.00010 Operator 11 53474 4861.27 11.9829 0.00000 Operator*Sample 77 31238 405.68 1.4907 0.03177 Repeatability 96 26125 272.14 Total 191 125058 %Contribution Source VarComp (of VarComp) Total Gage R&R 617.39 90.11 Repeatability 272.14 39.72 Reproducibility 345.25 50.39 Operator 278.47 40.65 Operator*Sample 66.77 9.75 Part-To-Part 67.75 9.89 Sample, Operator, & Interaction are Significant

  17. CTQ1 MSA Study Results *Conf Int not calculated with Minitab, Based upon R&R Std Dev

  18. CTQ1 MSA Study Results (Minitab Output) Dotplot of All Samples over All Sites Site 3 Site 1 Site 2 Site 4

  19. CTQ1 MSA Study Results (Minitab Session) Analysis of Variance for Site Source DF SS MS F P Site 3 37514 12505 26.86 0.000 Error 188 87518 466 Total 191 125032 Individual 95% CIs For Mean Based on Pooled StDev Level N Mean StDev -+---------+---------+---------+----- Site 1 48 824.57 15.38 (---*---) Site 2 48 819.42 22.11 (---*---) Site 3 48 800.98 20.75 (---*---) Site 4 48 840.13 26.58 (---*---) -+---------+---------+---------+----- Pooled StDev = 21.58 795 810 825 840 Site and Operator are closely related

  20. CTQ1 MSA Study Results (Minitab Output) X-bar R of All Samples for All Sites Discrimination Index is “0”, however can probably see differences of 5 Most of the samples are seen as “noise”

  21. X-bar R of All Samples for Site 4 CTQ1 MSA Study Results (Minitab Output) Mean differences are seen in X-bar area Most of the samples are seen as “noise”

  22. CTQ1 MSA Study Results – Process LinkageSite 2 Example MSA Study Results with Mean = 819.4 2002 Historical Process Results with Mean = 832.5 Selected Samples are Representative

  23. CTQ1 MSA Study Results – Process LinkageSite 2 Example MSA Study Results with Range = 17.92, Calc for Subgroup 2002 Historical Process Results with Range = 25.08 Calc for pt to pt When comparing the MSA with process operation, a large percentage of pt-to-pt variation is MS error (70%) --- a back check of proper test sample selection

  24. CTQ1 MSA Study Results – Process LinkageSite 2 Example • Key issue for Process Improvement Efforts is “When will we see change?” • Initial Improvements to A1 process were made • Control Plan Improvements to A1 process were initiated • Site 2 Baseline Values were higher than other sites • Small step changes in mean and reduction in variation will achieve goal • How can Site 2 see small, real change with a Measurement System with 70+% GR&R? Use Power and Sample Size Calculator with and without impact of MS variation. Lack of clarity in process improvement work, results in missed opportunity for improvement and continued use of non-optimal parameters

  25. CTQ1 MSA Study Results – Process LinkageSite 2 Example 2-Sample t Test Alpha = 0.05 Sigma = 22.23 Sample Target Actual Difference Size Power Power 2 2117 0.9000 0.9000 4 530 0.9000 0.9002 6 236 0.9000 0.9002 8 133 0.9000 0.9001 10 86 0.9000 0.9020 12 60 0.9000 0.9023 14 44 0.9000 0.9007 16 34 0.9000 0.9018 18 27 0.9000 0.9017 20 22 0.9000 0.9016 2-Sample t Test Alpha = 0.05 Sigma = 6.67 Sample Target Actual Difference Size Power Power 2 192 0.9000 0.9011 4 49 0.9000 0.9036 6 22 0.9000 0.9015 8 13 0.9000 0.9074 10 9 0.9000 0.9188 12 7 0.9000 0.9361 14 5 0.9000 0.9156 16 4 0.9000 0.9091 18 4 0.9000 0.9555 20 3 0.9000 0.9095 Simulated Reduction of Pt to Pt variation by 70% decreases time to observe savings by over 9X.

  26. CTQ1 MSA Study Results – Process LinkageSite 2 Example Benefits of An Improved MS • Realized Savings for a Process Improvement Effort • For A1, an increase of 1 number of CTQ1 is approximately $1 per ton • Change of 10 numbers, 1000 Tons produced in 1 month (832  842) • $1 * 10 * 1000 = $10,000 • More trust in all laboratory numbers for CTQ1 • Ability to make process changes earlier with R-bar at 6.67 • Previously, it would be pointless to make any process changes within the 22 point range. Would you really see the change? • As the Six Sigma team pushes the CTQ1 value higher, DOEs and other tools will have greater benefit

  27. Objectives • The Hidden Factory Concept • What is a Hidden Factory? • What is a Measurement System’s Role in the Hidden Factory? • Review Key Measurement System metrics including %GR&R and P/T ratio • Case Study at W. R. GRACE • Measurement Study Set-up and Minitab Analysis • Linkage to Process • Benefits of an Improved Measurement System • How to Improve Measurement Systems in an Organization

  28. Measurement Improvement in the Organization • Initial efforts for MS improvement are driven on a BB/GB project basis • Six Sigma Black Belts and Green Belts Perform MSAs during Project Work • Lab Managers and Technicians are Part of Six Sigma Teams • Measurement Systems are Improved as Six Sigma Projects are Completed • Intermediate efforts have general Operations training for lab personnel, mostly laboratory management • Lab efficiency and machine set-up projects are started • The %GR&R concept has not reached the technician level • Current efforts enhance technician level knowledge and dramatically increase the number of MS projects • MS Task Force initiated (3 BBs lead effort) • Develop Six Sigma Analytical GB training • All MS projects are chartered and reviewed; All students have a project • Division-wide database of all MS results is implemented

  29. Measurement Improvement in the Organization • Develop common methodology for Analytical GB training

  30. Final Thoughts • The Hidden Factory is explored throughout all Six Sigma programs • One area of the Hidden Factory in Production Environments is Measurement Systems • Simply utilizing Operations Black Belts and Green Belts to improve Measurement Systems on a project by project basis is not the long term answer • The GRACE Six Sigma organization is driving Measurement System Improvement through: • Tailored training to Analytical Resources • Similar Six Sigma review and project protocol • Communication to the entire organization regarding Measurement System performance • As in the case study, attaching business/cost implications to poorly performing measurement systems

More Related