HSBC Six Sigma Black Belt Training
This presentation is the property of its rightful owner.
Sponsored Links
1 / 401

HSBC Six Sigma Black Belt Training Analyse PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on
  • Presentation posted in: General

HSBC Six Sigma Black Belt Training Analyse. April 2006 Rev 1.0. Analyse Phase. Module 1. Recap of the Measure Phase. Module 2. Overview of the Analyse Phase. Module 3. Graphical Data Analysis. Module 4. Simple – Identify, Rank and Validate Key X’s - 5 Why - Cause and effect diagram

Download Presentation

HSBC Six Sigma Black Belt Training Analyse

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Hsbc six sigma black belt training analyse

HSBC Six Sigma Black Belt Training

Analyse

April 2006 Rev 1.0


Analyse phase

Analyse Phase

Module 1

Recap of the Measure Phase

Module 2

Overview of the Analyse Phase

Module 3

Graphical Data Analysis

Module 4

Simple – Identify, Rank and Validate Key X’s

- 5 Why

- Cause and effect diagram

- Multi-voting

Module 5

Validate the Vital Few

- One sample methods

- Two sample methods

- Chi-Square

Module 6

More Advanced – Identify, Rank and Validate Key X’s

- ANOVA

- Simple regression

Advanced – Identify, Rank and Validate Key X’s

- Introduction to design of experiments

Module 7

Module 8

Tollgate


Methodology overview

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice

Methodology overview

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:


Analyse phase1

Analyse Phase

Module 1

Recap of the Measure Phase

Module 2

Overview of the Analyse Phase

Module 3

Graphical Data Analysis

Module 4

Simple – Identify, Rank and Validate Key X’s

- 5 Why

- Cause and effect diagram

- Multi-voting

Module 5

Validate the Vital Few

- One sample methods

- Two sample methods

- Chi-Square

Module 6

More Advanced – Identify, Rank and Validate Key X’s

- ANOVA

- Simple regression

Advanced – Identify, Rank and Validate Key X’s

- Introduction to design of experiments

Module 7

Module 8

Tollgate


Measure phase objectives

Measure Phase Objectives

Measure Phase

  • It’s critical we first understand data demographics for our project to determine sampling characteristics and data collection requirements

  • We won’t repeat all of Measure, instead just cover

    • Measurement System Analysis (MSA)

    • Data Collection

    • Sample Size Determination

    • Process Capability

Before moving intoanalyse, let’s leverage our tools to betterexplore and explain our project data


Measure phase remove measurement variation

Measure Phase – Remove Measurement Variation

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice

Pocket guide (pages 41-45)


Measure phase remove measurement variation1

Measure Phase – Remove Measurement Variation

  • We completed Measurement System Analysis (MSA) last time

  • Review the details in the material on the intranet or on the course CD

  • Measurements should be

    • Precise/accurate

    • Repeatable

    • Reproducible

    • Stable over time

    • Have adequate resolution

Measure

Tollgate

A.Collect or create inputs

B.Determine the correct decision

4. Map and analysethe process

C.Identify the decision makers

5. Remove measurement variation & collect data

6. Determine process capability

D.Administer the assessment

E.Analyse the outcomes and take action

MSA is an essential first step to minimise measurementbias prior to sampling & data collection


Msa what is it

MSA – What Is It?

  • A measurement system is composed of all the components and the processes used to obtain quantitative information about a process characteristic

  • Typical components are

    • People – the assessors

    • Measurement tool(s) – the gage

    • Material – the actual item/process being measured

    • Method – procedures

    • Environment – external conditions

  • A measurement system analysis is the investigation into how all of these components work together to measure the information needed to understand the process

Unfortunately, these components can bring their own level of variation to the process. MSA is analysis and control of measurement variation


Msa review

Repeatability

Variation that occurs when repeated measurements are made of the same item under absolutely identical conditions

Reproducibility

Variation that occurs when different conditions are used to take measurements

MSA Review

OperatorB

You’re a smooth operator… Let’s seeif your friends are as smooth as you!

Ask me again, and I’ll tell you the same!

OperatorC

OperatorA

Repeatability

Reproducibility

Our measures need to be reliable


Msa review1

Step 1 – Create inputs

Decide on the outputs to be evaluated (inputs)

The inputs need to be

Representative

Equally represented in the set of outputs

Correct decision should not be too obvious

Sufficient sample size

Ensure 50% of inputs are defect-free

Determine number of inputs needed

Step 2 – Determine standard

Develop a standard

Have two credible decision makers inspect or review each input

Come to a consensus agreement with each other as to the correct disposition

MSA Review


Msa review2

Step 3 – Identify decision makers

Identify the person or persons who are going to participate in the study

These should be individuals who make the decisions within the process under study on a regular basis

Selected decision makers have to meet the following guidelines

Familiar process

Same location

Same time constraints

Step 4 – Administer

The structure of the assessment requires that each decision maker evaluate each item at least twice

Process

1st person evaluates all of the samples in trial 1

2nd person does the same

Once all the people have assessed, the samples are returned to the 1st person for an evaluation in trial 2

Then back to 2nd person

Be aware of bias

MSA Review


Msa review3

Step 5 – Analyse outcome and take action

Analyse outcome and take actions

Assess

Assessor effectiveness

Overall MSA

Biases

Actions include

Immediate corrections

Refined definitions

Training

Gage recalibration

Starting over

MSA Review


Measure phase collect data

Measure Phase – Collect Data

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice

Pocket guide (pages 22-52)


Measure phase collect data1

Measure Phase – Collect Data

Measure

Tollgate

A.Data demographics

4. Map and analysethe process

Remember, must complete MSA prior to collecting or analysing data!

5. Remove measurement variation & collect data

6. Determine process capability

B.Sampling

Data collection begins with the end in mind. “What do you want to know?”


Collect data sampling

Collect Data & Sampling

Measure

A1. Select what to measure

Tollgate

A2. Develop operational definitions

A.Data demographics

A3. Identify data sources

4. Map and analysethe process

A4. Prepare data collection form

A5. Implement and refine data collection

5. Remove measurement variation & collect data

B1. Sample types and terminology

6. Determine process capability

B2. Confidence

B.Sampling

B3. Sampling techniques

B4. Sample size formulas/calculators

Data demographics + sampling = Data collection


Select what to measure

Select What To Measure

  • Selecting what you need to measure is not always easy

  • Initially, you may have to simply count defects that show up in your process output (Y)

  • Then, go back to your process map and measure the performance of those process steps that seem to be contributing to output defects (suspect x’s)

  • Use a CTQ Tree format to identify measures

  • Use the XY Matrix to prioritise x’s

Application availability (uptime)

Systems

Funding source

Network availability (uptime)

Market instructions

Timely trades (Y)

Advisor

Origination

Destination

Equity of fixed income

Security

Lot size

Market

1st Step: What are you going to measure?


Develop operational definitions

Develop Operational Definitions

  • Saying that your team will count the number of defects in a service is easy. But what do you mean by “defect” or “service”?

  • Without having precise definitions for the things you’re trying to measure, different people will count different things in different ways

  • To avoid this confusion, you need to have operational definitions

    • A clear, understandable description of what’s to be observed and measured, such that different people taking or interpreting the data will do so consistently

2nd Step: Create clear and understandable data definitions


Identify data sources

There are two main sources of data available to the team

Data that is already being collected in your organisation and has been around for some time (usually called “historical” data)

New data that your team collects

Historical data can be handy, when you have it - it requires fewer resources to gather, it’s often computerised, and you can start using it right away

But be warned! Existing data may not be suitable if

It was originally collected for reasons other than process improvement

It uses different definitions

Data structure makes it hard to stratify (or database lacks sort capability)

Identify Data Sources

3rd Step: Where are you going to get the data?


Prepare data collection plan

What?Why?Who?How?When?Where?

MeasuresOperational definitionFormulaPurposeSingle person Collection methodDates/times Source

of the data responsible frequency

collection

Prepare Data Collection Plan

Actions taken to validate measurement system

Sampling information

Type of Data DiscreteContinuous(please circle one)

Sample size

Collection time period

4th Step: Now that you know what you want… how do we plan to get it?


Implement and refine data collection

Implement And Refine Data Collection

  • There are five steps in implementing and refining the data collection process

    • Review and finalise your data collection plans

    • Prepare the workplace

    • Test your data collection procedures

    • Collect the data

    • Monitor accuracy and refine procedures as appropriate

Final Step: Are you sure that your plan will work? Have you tested it?


Measure phase determine sample size

Measure Phase – Determine Sample Size

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice

Pocket guide (pages 47-51)


Sample types terminology

Sample Types & Terminology

  • Samples are either judgmental or statistical

    • A judgmental sample is selected based upon the opinion of the analyst and the results may be used to make inferences only about those items from within the sample

    • A statistical sample is randomly selected from the entire population and the results may be used to make inferences about the entire population

Judgmental vs Statistical sampling

Judgmental sample

Statistical sample

  • Sample is selected based on knowledge and experience

  • Only a subset of the population is included in the selection process

  • Sample is assumed to be representative of the population

  • Sample is selected randomly

  • Entire population is included in the selection process

  • Sample is representative of the population

1st Consideration – Do you need opinions or facts?


Sample types terminology1

Sample Types & Terminology

Sampling is the process of collecting only a portion of available data either from a static data group (population) or on an ongoing basis (process), and drawing conclusions about the total population when the process is stable (statistical inference)

  • Population (N): The entire set of objects or activities for a process

    • The mean (μ) is the arithmetic average calculated for a population

    • The standard deviation (σ) is calculated for a population

  • Sample (n): A group that is part or subset of a population

    • The mean (x) of a sample

    • The standard deviation (s) of a sample

The sample is a “window” into the population


Sample types terminology2

Sample Types & Terminology

Population approach

  • Make probability statements about the population from the sample

  • “I have 95% confidence that the mean of the population is between 1.5 and 2.5 min”

    Process approach

  • Assess the stability of the population over time

    • Are shifts, trend, or cycles occurring?

    • Special or common cause?

Population approach

Process approach

…the type of “window” you use depends on the population


Confidence level and precision

10000

8000

6000

4000

Sample size

2000

0

-2000

0

.05

.1

.15

.2

Precision interval

Confidence Level And Precision

  • Confidence is the probability that the actual population value being estimated will be contained within the precision interval of our estimate

  • The precision interval represents the total amount of sampling error that you should expect for any specific sample size

  • This chart shows the relationship between sample size and precision for estimating the proportion of defects in a transaction process (95% CL)

Sample size and precision interval

2nd Consideration – How precise do you need to be?


Sampling techniques

Population

Sample

X

X

X

X

X X X X

X

X

Randomsampling

X

X

X

X

X

X

Population

B

A

A

B

Sample

B

B

A

B

A

B

A A B B B C D D

Stratified

random sampling

B

D

D

D

C C

D

D

D

Sampling Techniques

  • Random

    • Sample is selected in a purely random fashion

    • Each unit has the same chance of being selected

  • Stratified random

    • The population is segmented into more than one layer (stratum) and items are randomly selected within each layer

    • Every item in the population has a chance (not equal) of being included in the sample


Sampling techniques1

X

X

Sample

Systematicsampling

X X X X

Preserve Time Order

09:3009:45 10:00 10:15

XXXXX XXXXXXX XXXXXX XXXXXX

Subgroupsampling

Sample

X X X X

Preserve Time Order

Sampling Techniques

  • Systematic

    • Samples are selected based on a pre-defined sequence and are selected as they’re being produced by the process

  • Subgroup

    • Sample n units every t hour (ex: 3 units every hour)

    • Calculate the mean (proportion) for each subgroup


Sample size formula

Sample Size Formula

Data

Variable/continuous

Attribute/discrete

2

2

1.96s

1.96

[p(1-p)]

n=

n=

d

d

n = ?

s = 24.03

d = 2

n = ?

d = .02

p = 5%

n= sample size

s = standard deviation

d = precision

p = proportion defective

1.96= 95% Confidence

Lastly, how much data do you need?


Hsbc six sigma black belt training analyse

Sample size n=54

Sample Size Calculators (JMP)


Measure phase determine process capability

Measure Phase – Determine Process Capability

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice

Pocket guide (pages 86-92)


Determine process capability

Determine Process Capability

  • We covered capability last time. You can review the details in the material on the intranet or on the Course CD, or refer to pages 86-93 in the Pocket Guide

  • Process capability measures how much variation there is in a process relative to customer specification

  • As your data collection improves the picture of your process, go back and validate the process capability relative to customer specifications

Measure

Tollgate

A.VOC

4. Map and analysethe process

B.Determine Y

5. Remove measurement variation & collect data

6. Determine process capability

C.Z, Sigma, Yield

Process capability is the metric that our customers feel


Determine process capability1

Determine Process Capability

Identify process

Define CTQ

Define unit, defect, & defect opportunity

Count units, opportunities, & defects

Discrete orcontinuous

data

Discrete

Continuous

Calculate defect rate:

Count defects per

million opportunities

Identify distribution set defect limits

calculate yield

Look up Sigma value

in table

Convert yield

into short term sigma value


Determine process capability2

Determine Process Capability

  • Step 1 – Use process measures to determine output (Y)

  • Step 2 – VOC data and analysis will determine customer specifications in terms of Y measure

  • Step 3 – Determine defects

    • Continuous data: Calculate z score and translate to sigma or yield

    • Discrete data: Calculate DPMO and translate to yield of sigma


Analyse phase2

Analyse Phase

Module 1

Recap of the Measure Phase

Module 2

Overview of the Analyse Phase

Module 3

Graphical Data Analysis

Module 4

Simple – Identify, Rank and Validate Key X’s

- 5 Why

- Cause and effect diagram

- Multi-voting

Module 5

Validate the Vital Few

- One sample methods

- Two sample methods

- Chi-Square

Module 6

More Advanced – Identify, Rank and Validate Key X’s

- ANOVA

- Simple regression

Advanced – Identify, Rank and Validate Key X’s

- Introduction to design of experiments

Module 7

Module 8

Tollgate


Methodology overview1

Methodology Overview

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice


Analyse phase objectives

Analyse Phase Objectives

Analyse phase

  • In Analyse, you will use data collected in the define & measure phase to

Module 4

Module 5

Module 6

Module 7

Identify sourcesof variation

Rank key causes of variation

Validate root causes

Analyse requires us to identify the “likely suspects…”


The paths for the analyse phase

The Paths For The Analyse Phase

6. Determine process capability

Simple

“<2 Sigma”

  • Graphical data analysis

  • Brainstorming

  • Five-whys

  • Cause and effect diagram

  • Multi-voting

  • Process analysis - waste elimination(covered in measure)

Measure

Tollgate

Process stable?

Common cause strategy

More advanced

“2-3.5 Sigma”

Yes

All tools above and

  • Validating the vital few

    • 1 sample methods

    • 2 sample methods

    • Chi-Square

    • 1 and 2 way ANOVA

    • Simple regression

No

Special cause strategy

  • Quick hits

  • Waves of RIP’s

  • Fix obvious problems

  • Just do it!

Advanced

“>3.5 Sigma”

All tools above and

  • DOE


Analyse phase3

Analyse Phase

Module 1

Recap of the Measure Phase

Module 2

Overview of the Analyse Phase

Module 3

Graphical Data Analysis

Module 4

Simple – Identify, Rank and Validate Key X’s

- 5 Why

- Cause and effect diagram

- Multi-voting

Module 5

Validate the Vital Few

- One sample methods

- Two sample methods

- Chi-Square

Module 6

More Advanced – Identify, Rank and Validate Key X’s

- ANOVA

- Simple regression

Advanced – Identify, Rank and Validate Key X’s

- Introduction to design of experiments

Module 7

Module 8

Tollgate


Methodology overview2

Methodology Overview

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify problem

  • Achieve consistency between

    • Problem statement

    • Business case

    • Goals & objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete team charter

4. Map and analysethe process

7. Identify sourcesof variation

10. Generatesolution ideas

13. Implement solution

2. Specify customer

requirements & MGP

5. Remove measurement variation & collect data

8. Rank key causesof variation

11. Select best fit solution

14. Monitor processand results

3. Complete high level process map

6. Determine process capability

9. Validate root causes

12. Test solution and confirm results

15. Replicate and share best practice


Module objectives

Module Objectives

By end of this module you should be able to

  • Catalogue the core tools available to graphically display data given the various combinations of continuous and discrete Y’s and x’s

  • Be proficient in executing each of the graphical analysis tools using JMP

  • Analyse tool output to identify sources of variation and candidates for further analysis


In the analyse phase

In The Analyse Phase

  • Primary activities to assist in identifying variation

    • Graphical data analysis

    • Brainstorming

    • Five-whys

    • Cause and effect diagram

Measure

Tollgate

A.Graphical data analysis

7. Identify sources of variation

B.Brainstorming

8. Rank key causes of variation

C.Five whys

9. Validate root causes

D.Cause and effect diagram

We have to first think “out of the box” before we can focus “inside the box…”


Identifying sources of variation

Identifying Sources Of Variation

  • Define provided focus and scope through the Charter

  • Measure provided further focus through data collection and calculation of process capability

  • Now, we need to identify the major sources of variation

  • We’ll “filter” the results to answer the critical question...

    • What vital few process inputs and variables (x’s) have the greatest impact on process performance (Y)?

We have so much data! We need good filters to sort through it all!


Identify sources of variation

Graphical data analysis

Brainstorming

Five- whys

Cause and effect

Process analysis

Key “nuggets”

(root causes)

Identify Sources Of Variation

  • Variation can be identified using a variety of tools

    • Graphical data analysis

    • Brainstorming

    • Five-whys

    • Cause and effect diagramming

    • Also... process analysis (covered in Measure Phase)

Our filters (tools) separate the vital few “nuggets” from the rest of the trivial many


Graphical data analysis and stratification

Stratification is a data analysis technique by which data are sorted into various categories

Through the identification of specific factors, one can surface suspicious patterns and uncover differences in processes

Important to “stratify the data” to focus and prioritise improvement efforts later in the Engineer Phase

Data stratification helps to identify the impact of each x on Y

Compares variation in Y with each individual x

Results identify critical x(s)

Concerned with the direction and magnitude of the relationship, not why one exists

Graphical Data Analysis And Stratification

Stratification is exploration


Graphical data analysis and stratification1

Stratification will point to the factors that have the greatest impact on a problem

May need to expand (or narrow) the problem and goal of your project

Establish priorities for further analysis

Provide clues regarding possible causes by comparing “good” and “bad”

Common factors used for stratification:

Type (What is occurring?)

Timing (When it occurs?)

Frequency (How often does it occur?)

Where (Where in the process or location?)

Who (Which business, department, employee, customer group?)

Graphical Data Analysis And Stratification

Goal of data stratification is to prioritise and focus efforts


Stratification example

Total trade cycle time

140

130

120

110

100

90

80

70

0

10

20

30

Months starting January 2003

Fixed income

Equity

140

140

130

130

120

120

110

110

100

100

90

90

80

80

70

70

0

10

20

30

0

10

20

30

Months starting January 2003

Months starting January 2003

Stratification Example


Graphical data analysis tool selector

Graphical Data Analysis Tool Selector

Discrete

Continuous

X

Y

Discrete/counts

Bar chart, Histogram, Pareto chart, Pie chart

Bar chart, Histogram, Pareto chart

Continuous

Box plot, Multi-variability chart

Scatter plot, Run chart*, Multi-variability chart

* When plotting Y against time - always use a Run Chart


Graphical data analysis tool selector1

Graphical Data Analysis Tool Selector

Discrete

Continuous

X

Y

Discrete/counts

Bar chart, Histogram, Pareto chart, Pie chart

Bar chart, Histogram, Pareto chart

Continuous

Box plot, Multi-variability chart

Scatter plot, Run chart*, Multi-variability chart

  • Discrete Y vs x: Generally, this type of data will focus on counts of defects or of specific output types

  • We can use simple tools like Pareto Charts and Pie Charts to identify any relationship between Y and x’s

* When plotting Y against time - always use a Run Chart


Pie chart pareto chart

160

100%

N= 160

140

90%

85%

120

75%

75%

100

55%

Number of errors

80

50%

60

40

25%

20

88

32

16

8

16

0

0

Typos

Other

Empty

fields

Missing

pages

Wrong

fieldsentries

Date Prepared

Type of error

Collected By

Date Source

Formula

Pie Chart, Pareto Chart

Pie chart

Trade error rate by brokerage offices

Pareto chart

Pareto chart of loan application errors5/12 to 5/13/96 - Raleigh office


Graphical data analysis tool selector2

Graphical Data Analysis Tool Selector

Discrete

Continuous

X

Y

Discrete/counts

Bar chart, Histogram, Pareto chart, Pie chart

Bar chart, Histogram, Pareto chart

Continuous

Box plot, Multi-variability chart

Scatter plot, Run chart*, Multi-variability chart

  • The Y is sometimes characterised as a discrete value, often as a defect (or non-defect)

  • The x being studied varies over a continuous range

* When plotting Y against time - always use a Run Chart


Histogram

Distributions

2004 YTD total

Histogram


Graphical data analysis tool selector3

Graphical Data Analysis Tool Selector

Discrete

Continuous

X

Y

Discrete/counts

Bar chart, Histogram, Pareto chart, Pie chart

Bar chart, Histogram, Pareto chart

Continuous

Box plot, Multi-variability chart

Scatter plot, Run chart*, Multi-variability chart

  • Continuous Ys and discrete x’s are one of the most common combinations

  • These Ys are often measures of cycle time or money

* When plotting Y against time - always use a Run Chart


Boxplot

Boxplot

One way analysis of 204 Avg P by R

Third quartile

Median

First quartile

Outliers


Graphical data analysis tool selector4

Graphical Data Analysis Tool Selector

Discrete

Continuous

X

Y

Discrete/counts

Bar chart, Histogram, Pareto chart, Pie chart

Bar chart, Histogram, Pareto chart

Continuous

Box plot, Multi-variability chart

Scatter plot, Run chart*, Multi-variability chart

  • Continuous data on both halves of the Y=f(x) equation allow for the strongest relationship

  • Scatter diagrams are most useful, but there are also more complex methods of linear and non-linear regression that can be applied to test the relationship

* When plotting Y against time - always use a Run Chart


Scatter plots

Scatter Plots

Strong positive correlation

Strong negative correlation

No correlation

Possible positive correlation

Possible negative correlation

Other pattern


Run charts

Run Charts

Checks processed per hour

A run chart plots results, in time order, with the X-axis describing the time component

and the Y-axis describing the measured variable

Checks processed

91011

91011121234

91011121234

Monday Tuesday Wednesday

X-Axis units could be dates, days, hours or other

time intervals. They could be just numeric counts

but more value might be added by proper annotation

  • If the mean and control limits are included, the run chart becomes a control chart

  • If the run chart is to be analysed for random variation, the median could be added


Run charts and analysis

A run is a group of data points that all fall on one side of the median (Exclude points that fall directly on the median)

Run Charts And Analysis

You can conclude that you have a special cause signal if

Numberof data points

You see fewer runs than this

You see moreruns than this

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

3

3

3

4

4

4

5

5

6

6

6

7

7

8

8

9

9

9

10

10

11

8

9

10

10

11

12

12

13

13

14

15

15

16

16

17

17

18

19

19

20

20


Run charts and analysis1

Too few runs

Shift

Median

Measurement

Measurement

Measurement

Measurement

Median

Too many runs

Trend

Measurement

Measurement

Measurement

Measurement

Median

Run Charts And Analysis


Run charts and analysis2

Run Charts And Analysis

Alternate (up & down)

Same value

Measurement

Measurement


Exercise profitability of futures trading customers

Exercise – Profitability of Futures Trading Customers

Instructions:

  • Y-variable: Net after servicing and assigned

    • This variable represents profits after certain overhead charges have been assigned to that customer

  • X-Factors to consider

    • Maintenance level per management

      • Low – Medium – High: Low means that a customer requires few bank resources, high means a great deal of intervention is required by the bank to maintain the relationship

    • Reactive/proactive?

      • Reactive – Proactive: Proactive customers call the bank for action, Reactive customers respond only to repeated calls by the bank

    • Clearing or execution type

      • Types of transactions (2 levels)

Goal: Practice using graphical analysis and exploration tools to identify the most and least profitable customer groups in HSBC Futures business

Time: 30 minutes


Relative frequency histograms of x factor counts

Maintenance level per management

Reactive/proactive?

Clearing or execution

Relative Frequency Histograms Of X-factor Counts


Bar charts of mean net after servicing and assigned

Bar Charts Of Mean (Net After Servicing And Assigned)


Chart of mean net after servicing assigned

Chart Of Mean (Net After Servicing Assigned)


Bar graph with x factor reordered

Bar Graph With X-factor Reordered


Bar chart with different x factors

Bar Chart With Different X-factors


Variability chart

Variability Chart

The variability chart permits analysis of more than two X-factors simultaneously!


Variability chart of raw data

Variability Chart Of Raw Data

Variability Chart for Net After Servicing and Assigned


Variability chart of standard deviation

2

1

Variability Chart Of Standard Deviation

Low Maintenance, Proactive and Clearing is the combination with the highest standard deviation. Some clients are very profitable, others much less so.


Graphical data analysis tool selector5

Graphical Data Analysis Tool Selector

  • Graphical data analysis and exploration is the core tool used to stratify data and begin the identification of sources of variation

  • The primary graphical analysis tools available are

Discrete

Continuous

X

Y

Discrete/counts

Bar chart, Histogram, Pareto chart, Pie chart

Bar chart, Histogram, Pareto chart

Continuous

Box plot, Multi-variability chart

Scatter plot, Run chart*, Multi-variability chart

  • Output from graphical analysis tools are our primary method of communication to the team and key stakeholders. Remember, pictures are worth a thousand words!


Hsbc six sigma black belt training analyse

Analyse Phase

Module 1

Recap of the Measure Phase

Module 2

Overview of the Analyse Phase

Module 3

Graphical Data Analysis

Module 4

Simple – Identify, Rank and Validate Key X’s

- 5 Why

- Cause and effect diagram

- Multi-voting

Module 5

Validate the Vital Few

- One sample methods

- Two sample methods

- Chi-Square

Module 6

More Advanced – Identify, Rank and Validate Key X’s

- ANOVA

- Simple regression

Advanced – Identify, Rank and Validate Key X’s

- Introduction to design of experiments

Module 7

Module 8

Tollgate


Methodology overview3

Methodology Overview

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify Problem

  • Achieve consistency between

    • Problem Statement

    • Business Case

    • Goals & Objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should Be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete Team Charter

4. Map and Analysethe Process

7. Identify Sourcesof Variation

10. GenerateSolution Ideas

13. Implement Solution

2. Specify Customer

Requirements & MGP

5. Remove Measurement Variation & Collect Data

8. Rank Key Causesof Variation

11. Select Best Fit Solution

14. Monitor Processand Results

3. Complete High Level Process Map

6. Determine Process Capability

9. Validate Root Causes

12. Test Solution and Confirm Results

15. Replicate and Share Best Practice


Module objectives1

Module Objectives

By end of this module you should

  • Be proficient in application of additional tools to identify sources of variation

    • Brainstorming

    • Five-whys

    • Cause and effect diagram

  • Be skilled at ranking and validating key x’s for project scenarios considered relatively simple


Identifying variation remember this slide

Graphical data analysis

Brainstorming

Five- whys

Cause and effect

Process analysis

Key “nuggets”

(root causes)

Identifying Variation – Remember This Slide?

  • Variation can be identified using a variety of tools

    • Graphical data analysis

    • Brainstorming

    • Five-whys

    • Cause and effect diagramming

    • Also... process analysis (covered in Measure Phase)

Our filters (tools) separate the vital few “nuggets” from the rest of the trivial many


Identifying variation brainstorming

Brainstorming is used to generate a lot of ideas quickly to identify potential causes

Brainstorming encourages creativity, involves everyone, generates excitement and energy, and separates people from the ideas they suggest

Methods

Rounds – Go around in turn, one item per turn, until everyone passes

Popcorn – Anyone calls out ideas, no order, until all ideas are out

Brainstorming guidelines

Give advance notice so people can leverage data or prior analysis from the Define and Measure Phases

Start with silent “think” time

Freewheel - don’t hold back

NO CRITICISM

Hitchhike - build upon ideas

The more ideas, the better

Post ideas

Use affinity diagrams (where appropriate) to model themes or patterns

Identifying Variation – Brainstorming


Identifying variation five whys

Identifying Variation – Five Whys

Why?

Why?

Sales reps

have poor time mngt skills

Sales rep.

forgot

50%

40%

Why?

Contract not

up-to-date

70%

Why?

Sales reps distracted by admin tasks

50%

Why?

Invoice keypunch

error

20%

Negotiations

started late

20%

Pricing error

85%

Customers went

bankrupt

5%

Problem

Shipment short

or damaged

5%

40% of customers didn't pay us for their last bill

Wait for

internal

approval

Incorrect bill

80%

40%

Customer given

rebate

5%

Different

interpretations

of contract

requirements

Customer changed

order

5%

Bill never

received

10%

15%

Ask “Why” Five Times


Identifying variation cause and effect diagram

Identifying Variation – Cause And Effect Diagram

  • This tool helps to

    • Identify demographic X’s

    • Reduce the number of potential X’s

    • Focus improvements

  • A fishbone diagram is a picture composed of lines and words designed to represent a meaningful relationship between an effect and its causes

  • It is also called an Ishikawa diagram

  • The fishbone diagram brings us closer to determining the vital few X’s that determine Y

  • It helps teams reach a common understanding of problems and exposes gaps and potential causes of defects


Identifying variation cause and effect diagram1

Cause

Effect

Categories

Problemstatement

Causes

Identifying Variation – Cause And Effect Diagram

  • Cause and effect diagrams provide a visual display of all possible causes of a specific problem

  • Use it when

    • You need to expand your team’s thinking to consider all possible causes

    • You need to communicate by compiling your root cause analysis and displaying in a graphical fashion

    • You need to illustrate a link between causes (x’s) and effects (Y)


Identifying variation cause and effect diagram2

Identifying Variation – Cause And Effect Diagram

  • Step 1Draw the outline of the fishbone diagram

  • Step 2Write the problem statement in the head (box) of the fishbone

  • Step 3Determine the major categories of the fishbone diagram using one of four methods

  • Step 4Begin with the major category that the team has identified as the most likely to produce the actionable root cause and begin to ask “Why?”. Ask “why” as many as five times, or until such time as the additional “why” would be absurd

  • Step 5Complete the entire fishbone for every category before moving on

  • Step 6Identify the most likely actionable root cause(s) and circle


Identifying variation cause and effect diagram3

Measurements

Materials

Men & women

Problem statement

Environment

Methods

Machines

Identifying Variation – Cause And Effect Diagram

  • Problem statement or “effect” goes in the “head”

  • Bones are labeled based on data analysis or brainstorming need

  • To ensure you’ve nailed all possible causes, consider using the “six m’s”

    • Measurements

    • Materials

    • Men & women

    • Machines

    • Methods

    • Mother nature (environment)


Identifying variation cause and effect diagram4

Identifying Variation – Cause And Effect Diagram

  • Brainstorm possible causes and attach them to appropriate categories

  • For each cause ask, “why does this happen?”

  • Analyse results. Are there any causes that appear repeatedly throughout the model?

  • As a team, determine the three to five most likely causes

  • Use this to determine which likely causes you will need to further verify, potentially using statistical tools

Measurements

Materials

Men & women

Cause

Why

Problem statement

Environment

Methods

Machines


Exercise cause and effect diagram

Exercise – Cause And Effect Diagram

Goal: Practice constructing a fishbone (Ishikawa) diagram

Instructions:

  • Break out in teams to construct the diagram

  • Use your process map, or data collected, or a process of your choice

  • Determine the major categories of the fishbone that relate to the effect

  • Develop the fishbone, completing at least one causal chain

  • Cloud the most likely root cause(s) for verification

  • Select a spokesperson from each team to present

Time:Teams:30 minutes

Report-out:15 minutes


Simple rank and validate key x s

Xs

Ys

Suspected

Suspected

Suspected

Causes

causes

Causes

1

2

3

4

5

6

Total

%

1

2

3

4

5

6

Total

%

Cr

Cr

VA

VA

NVA

NVA

VE

Total

VE

Simple – Rank And Validate Key X’s

Now that we have ideas, suspicions, and hypotheses about potential root causes ... How do we get organised?


Process variation

Process Variation

Process variation (predictor)

Process capability (response)

Output

S

I

C

P

O

x

x

x

x

Y

Not all x’s have the same contribution to Y...need to identify the vital few!


Rank and validate key x s multi voting

Rank And Validate Key X’s – Multi-voting

  • Multi voting is a tool to

    • Identify demographic X’s

    • Reduce the number of potential X’s

    • Focus improvement opportunities on the “vital few”

  • Faced with a list of ideas, problems, causes, etc., each member a group is given a set number of “votes”

  • Multi-voting steps

    • Step 1Brainstorm the potential causes

    • Step 2Summarize causes using affinity diagram

    • Step 3Write the causes in a format suitable for review by the entire team (flip chart, whiteboard, etc.)

    • Step 4Assign each team member a finite number of votes

    • Step 5Have each team member vote with their highest priority cause receiving the most votes, and so on

    • Step 6Summarize items with most votes and circle

Those items receiving the most votes get further attention/consideration


Rank and validate key x s frequency impact matrix

Rank And Validate Key X’s – Frequency/Impact Matrix

Impact

High

Low

Further investigation required

Further investigation possible

Many

Frequency

Further investigation possible

Few

Those items in the upper-left quadrant get the most further attention/consideration


Summary

Summary

  • After graphical data analysis has been performed, brainstorming, five-whys and cause and effect diagramming are tools appropriate for use in identifying potential sources of variation

  • Multi-voting and Frequency/Impact Matrix are tools to use to narrow the sources of variation to a concise list of probable causes or “likely suspects”

  • For the simplest project scenarios, these tools may be sufficient to validate root cause. More complex scenarios require use of statistical methods, covered next


Analyse phase4

Analyse Phase

Module 1

Recap of the Measure Phase

Module 2

Overview of the Analyse Phase

Module 3

Graphical Data Analysis

Module 4

Simple – Identify, Rank and Validate Key X’s

- 5 Why

- Cause and effect diagram

- Multi-voting

Module 5

Validate the Vital Few

- One sample methods

- Two sample methods

- Chi-Square

Module 6

More Advanced – Identify, Rank and Validate Key X’s

- ANOVA

- Simple regression

Advanced – Identify, Rank and Validate Key X’s

- Introduction to design of experiments

Module 7

Module 8

Tollgate


Methodology overview4

Methodology Overview

Define

Measure

Analyse

Engineer

Control

Tollgate

Tollgate

Tollgate

Tollgate

Tollgate

  • Clarify Problem

  • Achieve consistency between

    • Problem Statement

    • Business Case

    • Goals & Objectives

  • Obtain unbiased view of the requirements

  • Agree on project timeline

  • Develop macro view of process(es) involved

  • Establish baseline

  • Stratify problem or opportunity to a component level specific enough to analyse

  • Establish key areas of the process where the data are collected

  • Establish valid data collection plan

  • Remove or account for measurement variation

  • Finalise problem statement

  • Identify the critical factors driving the requirement(s) (Y’s)

  • Identify improvement impact

  • Find root cause(s) of variation

  • Generate solution

  • Develop and test improvements

  • Complete pilot testing

  • “Should Be” process

  • Develop cost/benefit

  • Develop/build implementation plan

  • Validate improvement

  • Establish new performance levels

  • Sustain good performance levels

  • Establish corrective & contingency action plan

  • Translate & transfer learnings

  • Celebrate!

Steps:

1.Complete Team Charter

4. Map and Analysethe Process

7. Identify Sourcesof Variation

10. GenerateSolution Ideas

13. Implement Solution

2. Specify Customer

Requirements & MGP

5. Remove Measurement Variation & Collect Data

8. Rank Key Causesof Variation

11. Select Best Fit Solution

14. Monitor Processand Results

3. Complete High Level Process Map

6. Determine Process Capability

9. Validate Root Causes

12. Test Solution and Confirm Results

15. Replicate and Share Best Practice


Module objectives2

Module Objectives

By end of this module you should be able to

  • Determine the confidence intervals around a process mean

  • Understand the fundamentals of statistical hypothesis testing

  • Use One Sample, Two Sample and Chi-Square hypothesis tests to detect statistically significant differences in project data to

    • Evaluate the performance of different strata within your project data

    • Confirm root causes or opportunities

    • Compare the effectiveness of potential or executed change


Finding root cause

Finding Root Cause

Black belt worldChallenges

Tools of choice

  • Confidence intervals

  • Hypothesis tests

    • t Test

    • Paired t Test

    • Two Sample t Test

    • Chi-Square Test

  • Need to compare past versus new actions

  • Determine whether or not real improvements were made

  • Isolate the cause of a problem or opportunity

  • To make sound and defendable decisions

  • To develop a reliable basis for making decisions

  • To use the right data and the right tools in supporting decisions


Overview of statistical methods

Overview Of Statistical Methods

Statistical method

Data Y

Data X

Typical Null Hypothesis

JMP platform

One-Sample

C

None

H0: mean = specified value

Distribution

Matched Pairs

C

2 continuous columns

H0: meanA = meanB

Matched Pairs

Two-Sample

C

N with 2 values

H0: meanA = meanB

Fit Y by X

ANOVA

C

N with at k>≥2 values

H0: meanA = … = meank

Fit Y by X or Fit Model

Simple Regression

C

C

H0: slope = 0

Fit Y by X or Fit Model

Chi-Square

N

N

H0: X and Y are independent

Fit Y by X

DOE

C

C or N

ANOVA type H0: Factors have no effect on Y

DOE, Fit Model


Types of questions types comparisons

Baseline

Benchmark

Regulatory requirement

Standard

Current process

Comparing before & after…

Before promotion

campaign

After promotion

campaign

Comparing separate groups

Cross-selling ratio Northern Region

Cross-selling ratio

Southern Region

Types Of Questions – Types Comparisons

Case 1

Case 2

Case 3

Region 1

Region 2

Region 3

Region n

Case 4


Outline of each method

Outline Of Each Method

  • Why do we care?

  • Use graphical tools to explain problem

  • What is the tool

  • When to use it

  • How to use it: Detailed steps

  • Pitfalls!

  • Recap


Validating the vital few

Validating The Vital Few

  • Confidence intervals from single sample

  • Testing hypotheses from a single sample

  • Two Sample versus matched comparisons

  • Chi-square


Why do we need confidence intervals

Why Do We Need Confidence Intervals?

  • Confidence intervals from single sample allow comparison of performance with baseline or target

  • Confidence intervals give assurance that change will make a positive difference

  • Confidence intervals give an indication of how solid the argument in favor of change is


Confidence intervals for mean

Confidence Intervals For Mean

  • A confidence interval for a (unknown population) mean is an interval of numbers within which the unknown value of the actual mean is believed to be

  • The chance that the confidence interval contains the actual mean is called the confidence coefficient or confidence level (usually expressed in %)

    • 95% is the most common confidence coefficient

    • 50%, 80%, 90%, 99% (and others) are also used depending on problem and data availability

  • Confidence intervals tell us how precisely we know the unknown population mean

    • Larger samples typically have narrower or tighter confidence intervals than relatively smaller samples

    • Smaller confidence coefficients have narrower confidence intervals than larger confidence coefficients

  • Confidence intervals are being calculated using sample means and standard deviation


Point estimate and confidence intervals

Point Estimate And Confidence Intervals

Lower confidence limit < point estimate < upper confidence limit

90% lower

confidence limit

90% upper

confidence limit

Point

estimate

95% lower

confidence limit

95% upper

confidence limit

Point

Estimate

99% lower

confidence limit

Point

estimate

99% upper

confidence limit

Higher confidence levels result in wider confidence intervals.The point estimate remains the same


Relative widths of confidence interval for mean

Sample size n=10

Sample size n=40

Sample size n=100

Sample size n=400

Relative Widths Of Confidence Interval For Mean

Sample mean and standard deviation being the same

  • Small sample sizes result in wider confidence intervals

  • Large sample sizes reduce interval length marginally less


Single point estimate versus confidence interval

Single Point Estimate Versus Confidence Interval

What is the regional Cross Selling ratio?

? ? ? ? ? ? ? ? ? ? ? ?

123

Answer 1: Based on 44 observations from different branches, the mean cross selling ratio is 2.87

Single point

estimate

2.87

123

Answer 2: Based on 44 observations from different branches, the mean cross selling ratio is from 2.73 to 3.0

Confidence

interval

2.87

122.733.0

Which Answer is the better – Point estimate or confidence interval?


Point estimate and confidence interval for population mean

Point Estimate And Confidence Interval For Population Mean

  • Point estimate of the sample mean is a single value most likely different from the population mean

    • For the n=44 sample size the point estimate was 2.87. Use this for communicating your best single value estimate of the mean

  • Confidence Interval for the sample mean provides a likely range of values for the population mean based on the sample mean

    • For the n=44 sample size the 95% confidence interval ranges from 2.73 to 3.0. This interval expresses the uncertainty inherent in using sample values for the unknown population values

  • The confidence level or confidence coefficient is the chance that the confidence interval is correct

    • In a 95% confidence interval, there is a 5% chance that the population mean is higher than 3.0 or lower than 2.73

Lower Cl = LowerConfidence limit < Population Mean < Upper Cl = UpperConfidence limit


Single point estimate of loan cycle time

7.14 days

Long-term mean cycle time from application to decision

5.2 days

Mean cycle time of 50 observations AFTER change

Single Point Estimate Of Loan Cycle Time

Based on long-term experience, the mean cycle time from date of application to decision was 7.14 days. This was too long

The loan application process has been stream-lined. The new process is working in a pilot program of the NYPAFL region

Based on a sample of 37 cycle times from date of application to decision, the new mean cycle time is estimated at 5.2 days

Is this a real improvement?


Confidence intervals for loan cycle times

7.14 days

Long-term mean cycle time from application to decision 7.14 days

90%LCI=3.5 days

90% UCI=7.0 days

95%LCI=3.1 days

Mean = 5.2 days

95% UCI=7.3 days

Confidence Intervals For Loan Cycle Times

Is this a real improvement?

95% confidence interval mean cycle time of 50 observations AFTER change

The 90% confidence interval is from 3.5 days to 7.0 days. The long-term mean cycle time of 7.14 days is outside that range

What about the 95% confidence interval for the mean?


Takeaways from loan cycle data

7.14 days

Long-term mean cycle time from application to decision 7.14 days

90%LCI=3.5 days

90% UCI=7.0 days

95%LCI=3.1 days

Mean = 5.2 days

95% UCI=7.3 days

Takeaways From Loan Cycle Data

  • Confidence intervals provide a more rigorous view of whether or not the new performance is truly different or better

  • In the previous example we have no clear-cut evidence that the improvement is real instead of normal variation

  • Both confidence intervals are relatively wide, because of the high variation in the data. One could look for additional observations to validate the improvement

Recommendation – Look further for causes of the long cycle times!


Single value versus confidence interval estimates

Single Value Versus Confidence Interval Estimates

  • Single value or point estimates

    • Most statistical summaries begin with a single value estimate

      • The mean time of the decision cycle is 5.2 days

      • The standard deviation of the decision cycle is 6.3 days

    • Single value estimates are imprecise because they are based on samples on not on values from the entire population

    • How precise are they?

  • Confidence interval estimates

    • Confidence intervals give a range of values

      • 95% CI for mean decision cycle time between 3.1 and 7.3 days

      • 95% CI for standard deviation is between 5.2 and 8.2 days

    • Confidence level is used to express the chance that CI is correct

      • 95% confidence means that there is a 95% chance that the interval for the mean cycle time is correct, i.e., is between 3.1 and 7.3 days


When to use point estimates and confidence intervals

When To Use Point Estimates And Confidence Intervals

  • Both point estimates and confidence intervals describe the data

  • Use point estimate in oral communications

  • Use confidence interval to communicate that the point estimate is subject to uncertainty. The width of the confidence interval expresses that uncertainty

  • Confidence intervals come with different confidence levels or coefficients. 95% Confidence Level most often used

    • 95% confidence levels implies that on average 19 out of 20 confidence intervals are correct, 1 out of 20 is wrong

    • 90% confidence level implies that on average 9 in 10 confidence intervals are correct and 1 out of 10 is wrong

“What is going on and how precisely do we know it?” Confidence intervals


How to get confidence intervals in jmp

2

Point Estimates

1

4

3

5

95% Confidence Interval

How To Get Confidence Intervals In JMP

Data is in Cross Sell n_44.jmp


Obtaining simple confidence intervals in jmp

Confidence intervals for mean and standard deviation

6

7

Obtaining Simple Confidence Intervals In JMP

  • Click on red triangle on variable bar Total X Sell

  • Select confidence interval

  • Select desired confidence coefficient (.95)


Supplemental information is my data normal

Normal Quantile Plot option adds a graph to the report that is useful for visualizing the extent to which the variable is normally distributed

If a variable is normal, the normal quantile plot approximates a diagonal straight line

This kind of plot is also called a quantile-quantile plot, or Q-Q plot

Supplemental Information – Is My Data Normal?


Exercise cycle time from application to decision in days

Exercise – Cycle Time From Application To DecisionIn Days

Goal: Practice computing and evaluating confidence intervals

Instructions:

  • Use the data from Cycle Time Application n_37.JMP

    The situation is described in the notes below

  • Use JMP to create appropriate graphical output. Then describe the data in your own words

  • Estimate the Mean Decision Cycle Time as represented by the data in column “Cycle time – date of application to decision” and its standard deviation

  • Interpret these two values. Do they make sense?

  • Interpret the 95% confidence interval for the Mean Decision Cycle Time

  • The current sample is based on 37 applications. A sample based on 200 loans with the same sample mean and Std Dev would result in a confidence interval that is the same length[___], shorter[___], or longer[___]

Time:15 minutes


Solution

Solution

  • Data are non-symmetric, “skewed to the right”, not Normally distributed; Median=3 is less than Mean=5.22

  • Point Estimates: Mean time=5.22, Std Dev = 6.34 days. These values seem long. Is improvement needed?

  • 95% C.I. for the mean time is from 3.10 to 7.33

  • More observations (assuming same Mean and Std Dev) would SHORTEN the confidence interval


Summary1

Summary

  • Confidence intervals are useful, because they express the precision (or lack thereof) of the sample estimates

  • Confidence intervals have been discussed primarily for the mean of continuous measurements or the proportion in a poll

  • Confidence intervals are applied to many other quantities such as standard deviations, regression coefficients, differences between means. The most common confidence level is 95%. Virtually all opinion polls use 95%

  • The length of the confidence interval can be influenced by the sample size and the confidence level

    • Larger sample sizes result in shorter confidence intervals

    • Lower confidence levels results in shorter confidence intervals and highererror rates


Validating the vital few1

Validating The Vital Few

  • Point estimation and confidence intervals from a single sample

  • Testing hypotheses from a single sample

  • Two Sample versus matched comparisons

  • Chi-Square

Baseline

Benchmark

Regulatory requirement

Standard

Current process

Case 1


Why do we need hypothesis tests

Why Do We Need Hypothesis Tests?

  • Diagnose and prove differences in performance based on the influence of Xs

  • Make sure that changes have impact in the direction and strength desired by customers

  • Take steps to ensure correct decision

  • Make sure that changes are real and that their impact can be maintained


Examples for hypothesis tests for a single mean

The Cross Selling ratio has been 2.7 per event

Recently a system to customize product offerings has been introduced and a promotional campaign built around it

Examples for Hypothesis Tests for a Single Mean

Current process

Proposed change

Did we make

a real change?

The time to decision for small loan applications takes 7.14 days from receipt of application

A new online process was intended to reduce that time. Data collected under the new system are available

A bank buying a loan portfolio, set the price assuming an average credit score of 700

Buyer and seller request a sample of loans to determine any price altering differences from the target 700


Types of change options

Y

Mean loan cycle time

Y

Cross-selling

Types Of Change Options

Desired goal

Direction of improvement

Typical examples

Relationship to CTQ

JMP: p-values

Smaller or lower is better

(One-sided)

Smallest value

Fastest processing

Lowest variability

Loan cycle

Time

Prob <t

Bigger or higher is better

(One-sided)

Highest amount

Biggest payback

Cross-selling

Profitability

Customer retention

Prob >t

Target is best

(Two-sided)

Midpoint of range or Maximum return at specific zone

Deal pricing

Prob >|t|

Y

Pricing

Target price


What are hypothesis tests

What are Hypothesis Tests?

  • Hypothesis tests are decision about a mean, standard deviation or other parameter value

  • The problem is reformulated into two so-called hypotheses:

    • Alternative hypothesis versus Null Hypothesis

  • The Alternative Hypothesis can be one-sided or two-sided

  • Based on the evidence contained in the data, either the Null Hypothesis or the Alternative Hypothesis is accepted

  • Probability of making the incorrect decision when

    • Null Hypothesis is true (alpha error)

    • Alternative hypothesis is true (beta error)

  • JMP uses p-values in decision rules


Steps in conducting a hypothesis test

Steps In Conducting A Hypothesis Test

Step 1: Problem question

Specify the improvement

goals. Determine what you are trying

to prove. In what direction (>, <, ≠)

do you expect the improvement?

Leads to Alternative Hypothesis

Step 2: Baseline

Specify the standard of comparison,

benchmarks, long-term means/Std Dev.,

standards, regulatory requirements,

customer specifications

Leads to Null Hypothesis

Step 3: Conduct test

Specify error probabilities (a, b),

minimum differences to detect,

sample size or data source,

run JMP

Results and interpretation


Developing hypothesis test for mean cycle time

Developing Hypothesis Test For Mean Cycle Time

Question: Did a recent change in the small business loan approval process reduce the mean cycle time?

Answers

No, mean cycle time did not change.

Yes, mean cycle time is lower than 7.14.

Hypotheses

Null Hypothesis

H0: mean = 7.14

Alternative hypothesis

HA: mean < 7.14

Smaller is better!


Hypothesis tests for the mean of cross selling

Answer

No, Cross Selling ratio did not change

Yes, Cross Selling ratio is now larger than 2.7

Hypotheses

Null Hypothesis

H0: mean = 2.7

Alternative Hyp.

HA: mean > 2.7

Hypothesis Tests For The Mean Of Cross-selling

Question: Did a recent promotional campaign for mortgage loans increase the Cross Selling ratio?

Bigger is better!


Three examples of statistical hypotheses

Three Examples Of Statistical Hypotheses

Example

Null Hypothesis H0: Standard, existing performance

Alternative hypothesis HA: Research question or claim: What we want to prove with the data

A

Cross-selling without customized offerings achieves a mean ratio equalto 2.7:H0: mean=2.7

Cross-selling with customized offerings increases average Cross-Selling to above 2.7:HA: mean >2.7

B

Loan application require a mean processing time of 7.14 days with old process:H0: mean=7.14 days

Loan application processed with the new online component require less time:HA: mean < 7.14 days

C

The loan portfolio has an average credit worthiness of 700:

H0: mean = 700

The loan portfolio has an mean credit worthiness different from 700:HA: mean ≠ 700


Alternative hypotheses of examples a b

Alternative Hypotheses Of Examples A & B

A. Right-sided HA (bigger is better!): Cross-Selling

  • The Null Hypothesis is a fixed value, e.g.

    • H0: Mean of X-sell = 2.7

  • The Alternative Hypothesis states the mean is greater than the one claimed in the Null Hypothesis, e.g.

    • HA: Mean of X-Sell > 2.7

  • The case where the mean is less than the value stated in the Null Hypothesis is of no interest with regard to the problem statement

    B. Left-sided HA (smaller is better): decision time of loan application

  • The Null Hypothesis is a fixed value, e.g.

    • H0: Mean time to decision= 7.14

  • The Alternative Hypothesis states the mean is less than the one claimed in the Null Hypothesis, e.g.

    • HA: Mean time to decision < 7.14

  • The case where the mean is greater than the value stated in the Null Hypothesis is of no interest with regard to the problem statement


Alternative hypothesis of example c

Alternative Hypothesis Of Example C

C.Two-sided HA (Target is Best!): Pricing a Loan Portfolio. The data are used to fix the final price of the loan portfolio?

  • The Null Hypothesis is H0: Mean credit score = 700

  • Null Hypothesis is often a target price

  • The Alternative Hypothesis states the mean is either less than or greater than the one claimed in the Null Hypothesis:

    • HA: Mean credit score ? 700

  • Two-sided alternative hypotheses can also be tested using confidence intervals


Hypothesis test with two sided h a

H0

H0

HA

HA

OR

700

700

Credit scores in portfolio have mean score <700

Credit scores in portfolio have mean score >700

Hypothesis Test With Two-sided HA

H0: Actual mean = Specified constant for mean credit worthiness is 700

HA: Actual mean ¹ Specified constant for mean credit worthiness is different from 700


Decisions and the courts

Defendant is not guilty

Defendant is guilty

Verdict is:“NotGuilty”

Good decision to acquit the innocent

Error: Guilty is declared innocent

Verdict is:“Guilty”

Error: Innocent is convicted

Good decision to convict the guilty

Decisions And The Courts

  • Error types are (1) Convict innocent and (2) Guilty is declared innocent


Errors and correct decision in hypothesis testing

Errors And Correct Decision In Hypothesis Testing

Data is evidence in hypothesis testing

True state of nature

Decision

HO is true

HA is true

Accept HO

Reject HA

Correct decision

Type II error  error probability

Accept HA

Reject HO

Type I error  error probability

Correct decision

Hypothesis testing allows us to quantify in advance the probability of an incorrect decisions(error probabilities  or .) The error probabilities can be controlled by taking appropriate sample sizes

Relatively large sample sizes reduce the error probabilities


Business impact and risk

Business Impact And Risk

Business impact of change

Magnitude of desired change

Process variability

Required sample size n

Required α

Required β

Critical and costly

small

small

moderate

0.05 or less

0.05 or less

Critical and costly

small

large

large

0.05 or less

0.05 or less

Neither costly nor critical

large

small

small

0.05 or larger

0.05 or larger

Neither costly nor critical

large

large

moderate

0.05 or larger

0.05 or larger


P value as decision rule

P-value As Decision Rule

  • The p-value expresses the strength of the data in keeping the Null Hypothesis. The smaller the p-value, the less likely are we to believe the Null Hypothesis

  • The significance levela is a threshold that allows us to decide whether to accept or reject the Null Hypothesis. a is the largest p-value at which the Null Hypothesis can be rejected

    • Decision rules with the p-value with a=0.05:

  • If p≤a = 0.05reject Ho and accept HA

    • If p>a = 0.05accept Ho and reject HA

  • If the Null Hypothesis can be rejected by this decision rule, the result is said to be statistically significant


  • Graphical representation of decision rule for right sided h a

    Graphical Representation Of Decision RuleFor Right-sided HA

    H0 mean

    Critical value of test statistic

    a = significance level

    1-a

    Region of test statistic to accept H0. p>a

    Region of test statistic to reject H0. p<a

    Acceptance region

    Rejection region


    Graphical representation of decision rule for two sided h a

    Graphical Representation Of Decision Rule For Two-sided HA

    H0 mean

    Lower critical value of test statistic

    Upper critical value of test statistic

    a/2

    1-a

    a/2

    Region of test statistic to reject H0. p<a

    Region of test statistic to accept H0. p>a

    Region of test statistic to reject H0. p<a


    When to use one sample hypothesis testing

    When To Use One Sample Hypothesis Testing

    • Single continuous variable Y (Cross Selling ratio, cycle time) needs to be analysed

    • External reference or standard (benchmark, long-term average, etc.) is available, so that sample of past process is not needed

    • Decision needs to be made to

      • Confirm that a change made a significant difference in mean or Std Dev of Y. (Mean cycle time has been significantly reduced!)

      • Confirm that a specific goal is met by the process. (Mean Cross Selling ratio meets the goal of 3.0!)

    • The strength of the evidence needs to be communicated to management. (Use p-value!)


    One sample hypothesis testing

    One Sample Hypothesis Testing

    • Single data column analysed by JMP’s Distribution platform

    • Null Hypothesis is external standard, benchmark or long-term average

    • Choose right-sided, left-sided, two-sided Alternative Hypothesis using problem statement as guide

    • Significance level often is a=0.05, but not always

    • Reject Null Hypothesis if p-value < a=0.05

    • Sample size requirements depend on a, power, the magnitude of the difference to detect, and the variability (std. dev.) of the outcomes (to be covered later)


    Testing cross selling improvements

    Testing Cross-Selling Improvements

    • Data of monthly Cross Selling ratios after improvements are available for n=44 branches. The ratios are shown in the data table CrossSell n_44.jmp. Start analysis using distribution platform. Note the historical average X-Sell is 2.7

    For these simple tests, the data consist of a single variable column (Total X-Sell)


    Obtaining and interpreting p values

    H0 entered

    Sample mean

    Two-sided p-value

    Right-sided p-value

    Left-sided p-value

    Conclusion: The sample mean is 2.866. The right-sided p-value = 0.0074 is less than a=0.05. Reject H0! Conclude that changes significantly increased the mean Cross Selling ratio above the historical mean of 2.7

    Obtaining And Interpreting P-values

    Historical average of Total X=Sell = 2.7


    Confidence intervals and two sided h a

    H0: mean=2.7

    Reject H0!

    95% CI does not include H0 mean

    2.7 2.8 2.9 3.0

    Confidence Intervals And Two-sided HA

    • A confidence interval for the mean can be viewed as a set of hypotheses that can not be rejected with the given data

    • Can not reject the Null Hypothesis H0 at level a(=0.05) as long as the associated mean as claimed by H0 is within the 100(1- a)% (= 95%) confidence interval

    • The 95% confidence intervals can be used to test two-sided hypothesis with a =0.05


    Finding the required sample size

    Finding The Required Sample Size

    Finding an Improvement in Cross-Selling

    • Historical cross-sell mean is 2.7

    • Desired improvement is at least an average of 0.3 to an average of 3.0

    • Currently the standard deviation of Cross-Selling between branches is 0.4

    • How large a sample is required,to find a change to 3.0 with a high probability of 0.95


    Sample size for simple hypothesis test in jmp

    3.0-2.7=0.3

    s=0.4

    b=0.05

    a=0.05

    b

    a

    Mean H0mean=2.7

    Mean HAmean=3.0

    Required Sample Size n=25

    Sample Size For Simple Hypothesis Test In JMP

     = probability reject H0 when it is true

     = probability reject HA when it is true

    Power = 1-  = prob to accept HA when true


    Practice example time to decision

    Practice Example – Time To Decision

    • The time between receipt of application to decision for a small business loan has been on average 7 days. A project is undertaken to reduce the average length of this process

    • How large a sample is required to show an improvement of 1 day with a high probability of 0.95

    • Use JMP to calculate the required sample size that would allow you to verify the improvement with high probability (= 0.95)

    • The process standard deviation is 8 days


    Changes in acceptance and rejection region as function of sample size

    Difference

    to detectSample size

    1 day833

    2 days210

    3 days94

    H0: mean =7.14

    R

    A

    Diff =1 day, n = 833

    A

    Diff =2 days, n = 210

    R

    R

    A

    Diff =3 days, n = 94

    Sample Mean

    3

    4

    5

    6

    7

    R

    A

    H0: mean = 7.14

    HA: mean < 7.14

    Accept H0!

    Improvement did not reduce cycle time

    Reject H0!

    Improvement reduced cycle time

    a =0.05, Power=0.95, s =8

    Changes In Acceptance And Rejection Region As Function Of Sample Size

    Large sample sizes give better discrimination for small and costly changes


    Pitfalls in hypothesis testing

    Pitfalls In Hypothesis Testing

    • Failure to write down the decision alternatives (hypotheses)

    • Insufficient sample size results in large error for the desired minimum difference to detect

    • Wrong p-value used

      • Be sure to select appropriate p-value (left-, right-, two-sided)

      • Select p-value from Alternative Hypothesis

    • Distribution is highly asymmetric

      • Less serious for test mean

      • Serious for Std Dev


    Summary of jmp requirements for one sample hypothesis tests

    Summary Of JMP Requirements For One Sample Hypothesis Tests

    • Y – Variable is continuous

      • Continuous as in time between application and decision

      • Continuous as a Likert scale [rate from 1,2,3,4,5]

    • No X-Variable used (except for separate analysis by groups)

    • JMP platform: Distribution

    • Input: single column of Ys

    • Output generated within:

      • Graphs

      • Moments: Mean, Std Dev

      • Test mean

      • Tests Std Dev


    Summary2

    Summary

    • Use One Sample tests whenever observations of performance have to be compared to standard, benchmark, or historical average

    • Use when improvements have to be validated

    • Use p-values less than 0.05 for determining statistical significance

    • Statistical significance does not always equate to practical relevance


    Validating the vital few2

    Validating The Vital Few

    • Estimation from a single sample

    • Testing hypotheses from a single sample

    • Two Sample versus matched comparisons

    • Chi-Square

    Case 2

    Cross Selling ratio

    Before promotion campaign

    After promotion campaign

    Case 3

    Cross-selling ratio Northern region

    Cross-selling ratioSouthern region


    Why do we need paired t tests and two sample tests

    Why Do We Need Paired T Tests And Two Sample Tests?

    • They allow comparison of the performance of two processes or two regions with each other without reference to an external standard

    • They allow before-after comparisons, such as same branch sales at two points in time

    • They allow us to pick the better of the two processes, if one is indeed better

    • They allow us to determine how solid the comparison between the two processes is


    Overview of matched and two sample t tests

    Overview Of Matched And Two Sample T Tests

    Previously we saw that

    • One Sample test comparing a sample mean against a specified value

      This section covers

    • Matched Pairs test comparing means based on pairs observations from same or similar sampling units

    • Two Sample tests comparing means based on two non-overlapping samples


    Some practical questions

    Two Sample questions

    Is there are difference in average Cross-Selling between regions NY/PA/FL and Rochester?

    Does the average time from application received to decision depend on whether the application is received complete?

    Does the average time from application to decision depend on customer type (2 categories)

    Matched Pairs question

    Does the new promotion campaign affect Cross-Selling at branches? Compare Cross-Selling at branches before aand after the promotion campaign.

    Some Practical Questions


    When to use matched pairs

    When To Use Matched Pairs

    • Whenever direct comparisons are possible

    • Whenever small changes need to be discovered and only a few sample units are available

    • Whenever two programs or changes can be evaluated on the same sampling unit (branch, employee, customer)

    Match pairs whenever you can, Two Sample if you must!


    Matched pairs design for 15 branches

    Matched Pairs Design For 15 Branches

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    B

    A

    15 branches with Cross Selling ratio recorded before (B) and 2 months After (A) pilot program to increase Cross-Selling was introduced. Each branch is compared with itself, after versus before


    Two sample design

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    2

    Two Sample Design

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    1

    32 observations from Group 1

    27 observations from Group 2

    n1=32

    n2=27

    Samples are taken from different processes or populations

    Two Sample designs are highly versatile for comparing two alternatives


    Hypothesis tests for comparing two means

    Hypothesis Tests For Comparing Two Means

    • Compare the means of two different processes, populations or approaches

    • Compare the means of the same process at two different time points

    • Better comparison than One Sample methods, because both processes are evaluated under similar conditions


    A comparison of comparisons

    A Comparison Of Comparisons

    One Sample comparisons

    Two Sample comparisons

    Matched Pairs comparisons

    Compare sample results with external standard

    Compare results samples from two different groups

    Direct comparison of sample at different times, etc.

    Always possible, if sample can be taken

    Usually possible, but with restrictions

    Not always possible

    Long-term average and sample results may have been collected under different conditions

    Sampling of two different groups is source of variability

    Sampling variability virtually eliminated due to direct comparison

    One group (long-term average, benchmark) is not sampled at all

    Sample size in two groups may differ

    Pairs of observations require One Sample size

    Use when resources are limited or when one needs to compare to standard

    Use when needed

    Use whenever possible


    Comparison of paired versus two sample data

    Two Sample

    Comparison based on different sampling units (8 branches, 4 with and 4 without pilot)

    Not a direct comparison

    Matched pair

    Comparison based on same sampling units (4 branches before and after pilot)

    Direct comparison is very sensitive

    Branch A

    2.9

    3.0

    Branch A with pilot

    3.0

    Branch B

    2.5

    2.65

    Branch B with pilot

    2.65

    Branch C

    2.8

    2.85

    Branch C with pilot

    2.85

    Branch D

    3.2

    3.25

    Branch D with pilot

    3.25

    Branch E without pilot

    2.9

    Branch F without pilot

    2.5

    Branch G without pilot

    2.8

    Branch H without pilot

    3.2

    Comparison Of Paired Versus Two Sample Data

    Cross-selling with and without pilot program: Does the pilot program increase Cross-Selling? Is the difference between with pilot and without pilot >0!

    Branch

    X-Sell

    Branch

    X-Sell before pilot

    X-Sell after pilot


    Matched pairs are more sensitive

    Matched Pairs Are More Sensitive!

    Two Sample results of SimTwoSamXsell.JMP

    Matched pair results of SimPairedXsell.JMP

    Mean difference = 0.0875

    8 branches

    p-value: Prob>t = 0.3322 > a=0.05

    Accept Null Hypothesis. Conclude that the pilot did not change Cross-Selling

    The 95% confidence interval for the mean difference is from -0.55 to 0.38

    Mean Difference = 0.0875

    4 branches

    p-value: Prob>t = 0.0177 < a=0.05

    Reject Null Hypothesis. Conclude that the pilot significantly changed Cross-Selling

    The 95% confidence interval for the mean difference is from -0.0113 to -.1637


    Hypothesis tests for comparing two means1

    Hypothesis Tests For Comparing Two Means

    • Depending on how the data was collected there are two types of mean comparisons

      • Matched Pairs analysed by the Matched Pairs platform in JMP

      • Two Sample test analysed by the Fit Y by X platform in JMP

    • Matched Pairs test assumes pairwise data on same or similar subjects (sampling units)

      • Same branch Cross-Selling in June and September

      • Cross-selling before and after promotional materials were introduced

    • Two Sample test assumes that all data were collected on different sampling units

      • Differences in average Cross-Selling between NY/PA/FL region and Rochester region

      • Difference in Cross-selling between medium-sized and large branch offices


    Research question hypotheses in two mean comparisons

    Research Question & Hypotheses In Two Mean Comparisons

    • Practical Question: Are the means of two groups A and B different?

    • This leads to the following Null Hypothesis

    • MeanA – MeanB = 0– Two means are equal, mean difference is 0

    • Versus one of the three the alternative hypotheses

    • MeanA – MeanB 0–Difference not equal to 0, two-sided

    • MeanA – MeanB < 0– Difference is less than 0, left-sided

    • MeanA – MeanB > 0– Difference is greater than 0, right-sided


    Matched pairs comparisons

    Matched Pairs Comparisons

    • Measures differences of observations on the same subject, object, branch, employee, customer. However, the two underlying observations differ by time period, program applied, incentive structure, and similar nominal categories

    • Direct comparisons are very efficient. Matched Pairs provide lots of information for relatively little sampling

    • Direct paired comparisons are not always possible

    • Examples of direct comparisons of performance

      • Cycle time before and after process changes

      • Cross-Selling before and after promotion campaign

      • Same branch sales one year apart

      • Employee performance subject to two different incentives schemes, but within the same branch (sampling unit is branch)


    Matched pairs looks at differences

    Matched Pairs Looks At Differences

    Step 1: Compare Cross-Selling of each branch with itself

    Cross-selling in a branch

    before promotion

    Cross-selling in a branch

    after promotion

    Difference in Cross-Sellingafter – before is of interest

    Step 2:

    analyse the differences

    Non-zero differences indicate change, impact, etc

    Zero difference implies no change


    Cross selling example for matched pairs

    Cross-selling Example For Matched Pairs

    Pairs 15 cross sell.JMP

    • Data are available on 15 branches that were involved in a pilot program to increase Cross-Selling. Data are available on Cross-Selling at each branch before the pilot program was introduced and for the second month after introduction of that program

    Did Pilot program increase Cross-Selling?


    Matched pairs test questions

    +0.05

    +0. 025

    0.00

    -0.025

    0.08

    Observed

    Difference

    0.06

    0.024

    0.04

    Difference

    0.02

    H0

    0.00

    -0.02

    -0.04

    1

    2

    3

    4

    5

    6

    7

    8

    9

    10

    11

    12

    13

    14

    15

    Branch

    Matched Pairs Test Questions

    Did Pilot program increase Cross-Selling?

    Average difference of Cross Selling ratio is 0.024. Is this significantly different from a zero difference?

    12 out of 15 differences are positive, indicating higher Cross Selling ratio after promotion. However, is this significant overall?


    Matched pairs cross selling in jmp

    Matched Pairs Cross-selling In JMP

    • Use Matched Pairs platform for data Pairs15Cross Sell.jmp

    • Select two columns containing pairs of observations. Enter “before” first to maintain sign throughout

    • Click OK


    Formulating the hypothesis tests for matched pairs cross selling

    Formulating The Hypothesis Tests For Matched Pairs Cross-selling

    • Sample size n=15 pairs of observations. Individual branches are the sampling unit. Only 15 branches were involved

    • Data columns consist of branch identifiers and of one column for Cross-Selling before and one column for after the pilot program was conducted

    • Research question: On average did the pilot program increase Cross-Selling?

    • Null Hypothesis:Mean(after)-Mean(before) = 0

    • Alternative hypothesis:Mean(after)-Mean(before) > 0(right-sided)

    • Use  = 0.05 as threshold for p-value


    Matched pairs cross selling results

    Mean differences based on 15 (before, after) pilot pairs is 0.02409

    This difference is statistically very significant (p=0.0005 < a=0.05)

    The difference is small enough to make one wonder whether the pilot program is worth extending to the other branches

    The 95% Confidence Interval for the mean difference is:

    Lower95% = 0.011 < actual mean difference < Upper95% = 0.037

    Need to evaluate proper course of action

    Small difference, but statistically significant!

    But does it really matter?

    Matched Pairs Cross-selling Results

    Did pilot program significantly increase Cross-Selling? Yes!

    Check table output for deciding on H0 or HA

    Graphical output is interesting, but can be ignored at first cut


    Jmp summary for matched pairs

    JMP Summary For Matched Pairs

    • Variable represented by two columns of Y each half of the pair

    • JMP platform: Matched Pairs

    • Input: two columns of Ys

    • Output

      • Difference between means

      • Confidence Interval for difference

      • P-values for two-sided, left-sided, right-sided hypothesis


    Matched pairs pitfalls

    Matched Pairs Pitfalls

    • In before/after recording of say Cross-Selling, factors other than the promotion program may have caused the increase. The statistics only know there is a difference, they do not tell you why!

    • Matched Pairs is not applicable when the sampling unit gets destroyed in the process of sampling. For example, in evaluating two programs to enhance customer satisfaction and retention, it may not be advisable to administer the two programs in succession. A Two-Sample approach, selecting two groups of customers, one for each program, may give more timely and less confusing results


    When to use two sample comparison

    When To Use Two Sample Comparison?

    Whenever two different processes or populations (regions, suppliers, programs) need to be compared and direct observations are impossible or difficult

    Match pairs whenever you can! Use two samples if you must!


    Two sample comparison of means

    Two Sample Comparison Of Means

    • Compare the mean of two independent populations using two separate and non-overlapping samples

      • Compare time to approval of loan for two comparable (but different) loan packages

    • Sampling of two groups, populations, or processes causes more variation in the sample results than with Matched Pairs

    • Examples of Two Sample comparisons

      • Compare Cross-Selling means of branches in region 1 & region 2

      • Compare the LOAN CYCLE TIMES before and after changes. Different loans before and after changes result in it a Two Sample process and not Matched Pairs

      • Different branches conduct 2 different programs to retain new customers. Compare the mean retention rates between branches assigned the different programs


    Two sample compares means of different samples

    Two Sample Compares Means Of Different Samples

    32 branches in region 1: NY/PA/FL

    27 branches in region 2: Rochester

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Cross-sell

    Sample Mean (Region 1)

    Sample Mean (Region 2)

    Region 1 sample mean of Cross Selling ratio

    Region 1 sample mean of Cross Selling ratio

    Two Sample compares means of samples from different groups!


    Two sample hypotheses

    Two-sample Hypotheses

    • Null Hypothesis typically assumes that there is no difference in mean

      • Mean(Region 1) – Mean(Region 2) = 0

    • Alternative hypothesis assumes a difference in mean either in one (left or right-sided) or both directions (two-sided)

      • Mean(Region 2)-Mean(Region 1) < 0(left-sided)

      • Mean(Region 2)-Mean(Region 1) > 0(right-sided)

      • Mean(Region 2)-Mean(Region 1)  0(two-sided)

    • Wording of hypotheses is very similar to Matched Pairs, but JMP platform and calculations are different

    Is the mean Cross Selling ratio of Region 1 different from that of Region 2?


    Cross selling example for two sample test

    Cross-selling Example For Two-sample Test

    • Data on Cross-Selling are available for 32 Region 1 branches and 27 Region 2 branches. The data are stored in two columns of CrossSellTwoSample.jmp

      • Column 1: Region in which branch is located

      • Column 2: “Total X-Sell” is the Cross-Sell Ratio for each listed branch

    • Null Hypothesis: There are no differences in mean Cross-Selling between regions

    • Alternative hypothesis: There is a difference. The direction of the difference is not specified. ?two-sided Alternative Hypothesis

    CrossSellTwoSample.jmp excerpt

    Is there a difference in mean Cross-Selling between Region 1 and 2 branches?


    Compare means

    2.810

    2.82

    2.81

    2.80

    2.79

    2.78

    2.77

    2.76

    2.75

    2.82

    2.81

    2.80

    2.79

    2.78

    2.77

    2.76

    2.75

    2.753

    NY/PA/FL

    Rochester

    Compare Means

    Are the two Cross-Sell ratios different? Yes, but are they significantly different?


    Two sample test questions

    Two Sample Test Questions

    Question

    • The scatter plot on the left and the individual 95% confidence intervals of Total X-Sell by Region show much overlap between the two regions. Does this eliminate the possibility that there is no significant difference between the means?

      Answer

    • No, overlapping data and confidence intervals do not eliminate the possibility of significant mean differences. However, in this case there are no significant differences – as shown next with p-values

    2.52.62.72.82.93.0

    NY/PA/FL

    Rochester

    95% confidence intervals for mean cross-sell ratio


    Cross selling two sample test in jmp

    Cross-selling Two Sample Test In JMP

    1

    1.Select Fit Y by X platform for data in CrossSellTwo Sample.jmp.

    2.For Two Sample t Test one needs

    • Continuous Y variable: Total X-Sell

    • Nominal X factor: Region

    2

    Is there a difference in mean Cross-Selling between Region 1 and 2 branches?


    Two sample hypothesis test

    Two Sample Hypothesis Test

    • Fit Y by X first shows scatter diagram

    • From submenu: “Means/ANOVA/t Test”

    • Mean diamonds are added to scatter diagram

    • Relevant numerical output is on next page

    3

    4


    Fit y by x output for two sample t test

    5

    Other output omitted!

    Fit Y By X Output For Two Sample T Test

    • 5. t Test shows

      • Estimated difference = -0.057

      • Lower 95% confidence = -0.302

      • Upper 95% confidence = 0.188

      • Two-sided p-value =0.6423

    Is there a significant difference in mean Cross-Selling between region 1 and 2 branches? no!

    There is no statistically significant difference! The mean of region 1 region is not different from that of region 2. p=0.6423>0.05 and the CI for difference includes 0 (-.3 to .19)


    Testing for unequal variances

    Testing for Unequal Variances

    Testing for unequal variance is performed for two reasons

    1a. Do the two processes have the same variation? The process with lower variation is the more consistent.

    1b. Are the returns of two investment sector equally (or unequally) certain? The variance is here a measure of uncertainty or risk. The lower the risk, the lower the variance of the returns.

    2. The standard Two-Sample t Test assumes that the variation of the underlying processes is the same. If the variation is statistically different as determined by the Levene Test, tests the differences between Means using the Welch ANOVA t-Test.


    Cross selling testing for unequal variances

    6

    7

    Cross Selling: Testing for Unequal Variances

    Fit Y by X

    Test for Unequal Variances:

    The Levene test for unequal variances is preferred in Six Sigma.

    p-value = 0.8281 -> Variances are EQUAL.

    Test for Equal Means:

    Welch test takes unequal variances into account.

    p-value = 0.6375 -> Means are EQUAL

    CrossSellTwoSample.jmp


    Compare mean cycle time of complete and incomplete applications

    Compare Mean Cycle Time Of Complete And Incomplete Applications

    Do complete applications have a shorter mean cycle time to decision?

    X Y

    Scatterplot shows overlap of observations in groups. Group mean diamonds do not overlap

    Clear separation of mean diamonds indicates significant difference of group means

    SmallBusinessData.jmp excerpt


    A cycle time two sample t test results

    1

    2

    3

    4

    95% individual CI for means

    12.4

    5.7

    5791113

    FALSE

    days

    TRUE

    A-cycle Time Two-sample T-test Results

    X-Factor: Application Complete?

    1.The average difference between the true and false levels is 6.74

    2.A 95% confidence interval for mean difference is between 5.3 and 8.2 days

    3.The p-value for testing no difference in means versus that there is a difference is p= 0.0000Null Hypothesis should be rejected. The difference is significant either at a one-sided or two-sided level

    4.The Means table shows means and confidence intervals for 158 incomplete and 581 complete applications in the data set.

    Incomplete applications significantly delay the average decision time!They approximately double the cycle time


    A cycle time levene test for unequal variances

    Small Business Data.jmp

    The Levene test has a p-value <0.0001.

    The variances (standard deviation) between the two levels are significantly different. Need to check Welch test.

    Factor: Applications Complete?

    Levels: TRUE, FALSE

    Large sample sizes bring out significant results.

    Welch test for unequal MEANS is also very significant with p<0.0001. However, this is the same result obtained with assuming equal variances. But this result is proper.

    A - Cycle Time: Levene Test for Unequal Variances


    Jmp requirements for two sample comparison

    JMP Requirements For Two Sample Comparison

    • Y – Variable is continuous

    • Two Sample comparison: X-Variable is Nominal with two values

    • JMP platform: Fit Y by X

    • Input: single column of Ys, single column of Xs

    • Output generated within:

      • Means of groups, difference of two means, Std Dev

      • Test mean difference for equal variances in each group

      • Test mean difference for uneuqal variance in each group


    Exercise compare loan approval time in hours

    Exercise – Compare Loan Approval Time In Hours

    Goal: Practice Two Sample hypothesis tests

    Instructions:

    • Changes in loan approval policies have been introduced in 8 branches of one district. Management wants to know whether these changes shortened the loan approval time

    • Situation A: Data on loan approval times were collected in the same 8 branches

      • Before changes in loan approval

      • After changes in loan approval

    • Situation B: Data on loan approval times were collected on 8 branches with the process changes and 8 different branches without the process changes

    • Identify which of the two situation is a Matched Pairs problem and which is a Two-Sample problem

    • Find out whether the changes in loan approval times with and without the process changes are significant. Give as complete an answer as you see fit

    Time:Teams:30 minutes

    Report out:15 minutes


    Solution to compare loan approval times

    Solution To Compare Loan Approval Times

    • Situation A and B are different in the way data were collected. Situation A is the standard “before” and “after” data collection and needs to be analysed using Matched Pairs. Situation B has data from 8+8=16 different branches and is a Two Sample problem

    • A: Null Hypothesis: Mean(after) = mean(before) versus the Alternative Hypothesis mean(after)<mean(before)

    • B: Null Hypothesis: Mean(change) = mean(without) versus the Alternative Hypothesis mean(change)<mean(without)

    • Hypotheses A and B are written as left-sided so therefore look for left-sided p-values

      • Matched Pairs p-value=0.0082: difference is significant

      • Two Sample p-value=0.0256: difference is significant

    • Same average difference (-0.9875) for both A & B, but higher p-value for Matched Pairs. Conclusion is the same (in this case!)


    Summary3

    Summary

    • Difference between Matched Pairs and Two Sample is the result of the way data were collected or are available

      • Matched Pairs: same branch sales a year apart

      • Two Sample: branch sales by two regions

    • Analysis differs greatly

      • Matched Pairs: use Matched Pairs platform

      • Two Sample: use Fit Y by X

    • Matched Pairs is more sensitive because of a more direct comparison

    • Two Sample does can be applied to situations when Matched Pairsdata could not be collected or are to difficult to collect

    • Hypotheses are formulated similarly


    Validating the vital few3

    Validating The Vital Few

    • Estimation from a single sample

    • Testing Hypotheses from a single sample

    • Two Sample versus matched comparisons

    • Chi-Square

    Internet banking

    use (X2)

    Age or income

    group (X1)


    Chi square tests for independence of x and y

    Chi-square Tests For Independence Of X And Y

    • When to use a Chi-Square test

    • Association does not measure cause – effect, merely an empirical association

    • Interpretation of the same association may depends on who interprets the data

    • Chi-Square test in JMP

    • Interpreting the results

    • Assumptions and extensions (Simpson’s paradox)

    • Conclusions


    Why we need chi square analysis

    Why We Need Chi-square Analysis

    • Data often come as percentages or frequencies reflecting a performance level

    • In analysis of questionnaires, find association between questions with categorical or ordinal outcomes

    • In analysis business data, find association between categorical demographic, socio-economic and similar characteristics

    • In production of services, relate categorical performance variables to categorical input variables

    Categorical data are often easier or cheaper to obtain. They often give sufficient information to analyse a problem


    Practical questions

    Practical Questions

    • Is internet bank usage associated with income level?

    • Is loan default associated with employment status?

    • In futures trading is proactive/reactive related with maintenance level by management?

    • Is viewing of TV programs associated with financing needs?

    • Is market location associated with preference of banking services?

    • Is attitude towards online transactions dependent on gender and age?


    Contingency tables cross tabulations and chi square test in jmp

    Contingency Tables, Cross Tabulations And Chi-square Test In JMP

    • Y – Variable is nominal measuring a categorical characteristic

    • X – Variable is nominal measuring a different categorical characteristic

    • Frequency is a variable that counts the number of occurrences of each (X, Y) combination

    • X and Y – Variable require each columns of possible outcomes

    • Frequency is a numeric, continuous column

    • JMP platform: Fit Y by X (or Fit Model for more complicated models)

    • Input: single column of Ys, single column of Xs, single column of frequencies

    • Output generated within:

      • Contingency table with counts, also select expected frequencies and cell Ch^2

      • Total, row, and column percentages if desired

      • Mosaic plot as a graphical representation

      • Pearson ChiSquare statistic with p-value


    When to use chi square

    When To Use Chi-square

    • Whenever questionnaires are analysed

    • Whenever categorical data need to analysed for associations

    • When numerical variables are reported as categorical variables

      • Income often is reported in categories [<30000, 30001-50000, 50001+]

    • Chi-square measures observed association

    Chi-square is used to validate conclusions about cause and effector assumptions about performance


    Savings book preference versus demographics

    Savings Book Preference Versus Demographics

    • Is the proportion of people who dislike the new savings book the same in all three demographic groups?

    Mosaic plot

    1.00

    0.75

    Like

    Like

    Like

    Like

    Preference

    0.50

    Overall proportions

    0.25

    Dislike

    Dislike

    Dislike

    Dislike

    0.00

    Men

    Women

    Children

    Person


    Savings book preference versus demographics1

    Savings Book Preference Versus Demographics

    • Problem Scenario

      • A bank has introduced a new savings book with a particular appeal. Respondents are classified into men, women, children 10 to 18 years. Each respondents having received a detailed explanation of the features, rates the savings book as either “like” or “dislike”

    • Is there an association (relationship) between person and preferences for the new savings product?

    • Data are in SaveBookPrefer.jmp

    Person by preference

    Count

    Dislike

    Like

    Row totals

    Men

    17

    24

    41

    Women

    19

    63

    82

    Children

    37

    68

    105

    Column total

    73

    155

    228


    Expected cell frequencies assuming independence

    SaveBookPrefer.jmp

    Person by preference (Expected cell frequencies are in italics)

    Count, Expected

    Dislike

    Like*

    Row totals

    Men

    17,13.1272

    24,27.8728

    41

    Women

    19,26.2544

    63,55.7456

    82

    Children

    37,33.6184

    68,71.3816

    105

    Column total

    73

    155

    228

    Independence of variables (people, preference), if observed cell frequencies and expected cell frequencies are close

    * Expected cell frequencies for Like are computed similar to those of Dislike

    Expected Cell Frequencies Assuming Independence

    • Expected cell frequency of cell(i,j)

      • E(i,j) = [total row (i) × total column (j)]/ overall total

    • Under the Null Hypothesis assumption, the expected number of men who dislike the savings book should be:

      • E(1,1) = [41 × 73]/228 = 13.1272

      • Compare this to the observed frequency n(1,1)=17


    Cell chisquare and pearson chisquare

    Cell-ChiSquare And Pearson ChiSquare

    For each individual cell (i,j) JMP calculates a Cell-ChiSquare. Small values support the Null Hypothesis

    The SUM of all cell-ChiSquares represents the test statistic from which the p-value is calculated

    Chi-Square test statistic is the sum of all squared differences between observed and expected cell frequencies divided by expected cell frequencies to make them more comparable. This is the Pearson Chi-Square test


    Contingency tables in jmp

    Loading raw data into JMP Table

    1

    Person by preference

    Count

    Dislike

    Like

    Men

    17

    24

    Women

    19

    63

    Children

    37

    68

    Contingency Tables In JMP

    2

    Preassigned frequency role!

    Choice of X and Y does not matter for statistical test, but may be important for conclusions

    3


    Contingency table output in fit y by x

    Contingency Table Output In Fit Y By X

    Preference

    4

    Count, expected

    Dislike

    Like

    Row totals

    Men

    17

    13.1272

    1.1426

    24

    27.8728

    0.5381

    41

    Men/dislike

    Women

    19

    26.2544

    2.0045

    63

    55.7456

    0.9440

    82

    Person

    Count

    17

    Children

    37

    33.6184

    0.3401

    68

    71.3816

    0.1602

    105

    Expected

    13.1272

    Cell Chi^2

    1.1426

    Column total

    73

    155

    228


    Chi square significance test

    Test

    ChiSquare

    Prob>ChiSq

    Likelihood ratio

    5.227

    0.0733

    Pearson

    5.130

    0.0769

    Chi-square Significance Test

    5

    Pearson ChiSquare p-value: The Pearson is the standard p-value for tests this hypothesis

    In this example Prob>ChiSq=0.0769 > 0.05

    Null Hypothesis can not be rejected

    One can not reject that the proportions of men, women and children who prefer the new savings book is different

    There is not enough evidence to say with statistical significance that the proportion of likes and dislikes changes by type of person, OR, that type of person and preference (like, dislike) are associated


    New example stratification of cross classified data

    New Example – Stratification Of Cross-classified Data

    Stratification by income variable!

    Combined table of strata 1 and 2 – FreqALL

    STRATA 1: High income respondents -FreqHIGHinc

    Personal loan

    Personal loan

    Count,expected

    No

    Yes

    Count,expected

    No

    Yes

    No

    8453.5385

    4878.4615

    132

    No

    419.6842

    4024.3158

    44

    Refinanced mortgage

    Refinanced mortgage

    Yes

    3262.4615

    12291.5385

    154

    Yes

    3014.3158

    217.6842

    32

    116

    170

    286

    34

    42

    76

    STRATA 2: Low Income Respondents – FreqLOWinc

    Personal loan

    Count,expected

    No

    Yes

    No

    8034.3619

    853.6381

    88

    Refinanced mortgage

    Yes

    247.6381

    12074.3619

    122

    Data are in LoanRefinChiStrata.jmp

    82

    128

    210


    Interpretation exercise

    1.00

    1.00

    0.75

    0.75

    Like

    Yes

    Yes

    Mortgage refinanced

    0.50

    Mortgage refinanced

    0.50

    0.25

    No

    0.25

    No

    Dislike

    0.00

    0.00

    NO

    YES

    NO

    YES

    Personal/Auto loan

    Personal/Auto loan

    Interpretation Exercise

    • In conjunction with the frequency tables from the previous slide, interpret the result of the mosaic plots

    • Compare the ease with which tables and mosaic plots lend themselves to interpretation

    STRATA 1: High income respondents

    1.00

    0.75

    Yes

    Mortgage refinanced

    0.50

    0.25

    No

    0.00

    NO

    YES

    Personal/Auto loan

    Combined table: ALL respondents

    STRATA 2: Low income respondents


    New example 2 stratification with counter intuitive results

    New Example 2 – Stratification with Counter-Intuitive Results

    Overall results of package services

    • Cross tabulation of MessEx and Bagone On Time and Late frequencies. 500 packages were sent with each Package Service. No significant association can be detected. The p-value is 0.525 and the Mosaic plot shows the proportion of “late” packages to be about even. Also Expected and Observe frequencies are very close

    On time performance

    1.00

    Count,Expected

    Late

    On time

    0.75

    Bagone

    230225

    270275

    500

    On time

    On time performance

    0.50

    Mess Ex

    220225

    280275

    500

    Package company

    0.25

    Late

    450

    550

    1000

    0.00

    Bagone

    Mess Ex

    Package company

    Data are in PackageService.jmp


    Stratification by regular and express mail

    Stratification by Regular and Express Mail

    500 express mail packages

    500 regular mail packages

    On time performance

    On time performance

    Count,Expected

    Late

    On time

    Count,Expected

    Late

    On time

    Bagone

    7054

    3046

    100

    Bagone

    160144

    240256

    400

    Package company

    Package company

    Mess Ex

    200216

    200184

    400

    Mess Ex

    2036

    8064

    100

    270

    230

    500

    180

    320

    500

    ChiSquare = 12.882 with p=0.000

    ChiSquare = 13.889 with p=0.000

    1.00

    1.00

    On time

    0.75

    0.75

    On time

    On time performance

    On time performance

    0.50

    0.50

    Late

    0.25

    0.25

    Late

    0.00

    0.00

    Bagone

    Mess Ex

    Bagone

    Mess Ex

    Package company

    Package company

    Which package service has better regular mail? Which service has better express mail?


    Pitfalls

    Pitfalls

    • Large data sets often result in significant Chi-Square tests, even though practically there is little difference between the proportions

    • Simple Chi-Square does only two variables at a time. More variables can be added, but the interpretation becomes more advanced (Need to use Fit Model)

    • Stratification is a method to break up the table. But there may not be sufficient data for some of the categories


    With chisquare sample size matters a lot

    With ChiSquare Sample Size Matters a Lot

    Service A

    Service B

    Service A

    Service B

    On time

    49

    51

    On time

    4,900

    5,100

    Late

    51

    49

    Late

    5,100

    4,900

    Total sample size n=200

    ChiSquare=0.8 with p=0.8, NOT significant!

    Total sample size n=20,000

    ChiSquare=8.0 with p<0.01, Significant!


    Exercise consumer complaints

    Exercise – Consumer Complaints

    Goal: Practice Two Sample hypothesis tests

    Instructions:

    • The frequencies of consumer business transactions and the frequency resulting in complaints are in the notes below

      • Using ConsumComplainBank.jmp, perform the analysis in JMP and discuss the results

      • Does the complaint rate of frequency depend on the sector?

        • State the Null Hypothesis in English

        • Tests whether it can be rejected using the appropriate p-value

        • State a final conclusion

        • Give as complete an answer as you see fit

    Time:Teams:15 minutes

    Report out:15 minutes


    Contingency or cross classification tables

    Contingency or Cross-Classification Tables

    • Individual observations classified according to two nominal variables

      • Nominal (or ordinal, categorical) variable (X) with r categories

      • Nominal (or ordinal, categorical) variable (Y) with c categories

    • Assign subjects to one of r x c categories

    • Number of sampling units large (n>100)

    • Example shows contingency table of Internet Banking use by two income levels

    Internet banking

    Count

    None

    Somewhat

    A lot

    Row total

    50<

    62

    51

    34

    147

    Income level

    51+

    37

    62

    64

    163

    Column total

    99

    113

    98

    310


    Solution1

    Solution

    • Both variables transaction type (Bank, Electronic, PC, Radio&TV) and complaints (yes, no) are nominal

    • Null Hypothesis is that relative frequency of complaints does not depend on transaction type

    • Use Pearson Chi-square value and associated p-value to test Null Hypothesis. Here p=0.55 and the Null Hypothesis can not be rejected

    • The conclusion is that the rate of complaints on these four types of business transactions is statistically about the same


    Summary4

    Summary

    • Chi-Square is used to test whether the proportion of defectives across several groups, regions or other factor levels is the same or whether there are significant differences

    • Chi-Square is used to confirm cause and effect relationships between categorical factors

    • Chi-Square is used to test association between demographic and other variables

    • Chi-Square can be refined by stratification, if a proper variable is available

    • Chi-Square should have sufficient data, preferably frequenciesof 5 or more, for the p-value of the Pearson Chi-Square to be valid


    Hsbc six sigma tool selector

    HSBC Six Sigma tool selector


    Analyse phase5

    Analyse Phase

    Module 1

    Recap of the Measure Phase

    Module 2

    Overview of the Analyse Phase

    Module 3

    Graphical Data Analysis

    Module 4

    Simple – Identify, Rank and Validate Key X’s

    - 5 Why

    - Cause and effect diagram

    - Multi-voting

    Module 5

    Validate the Vital Few

    - One sample methods

    - Two sample methods

    - Chi-Square

    Module 6

    More Advanced – Identify, Rank and Validate Key X’s

    - ANOVA

    - Simple regression

    Advanced – Identify, Rank and Validate Key X’s

    - Introduction to design of experiments

    Module 7

    Module 8

    Tollgate


    Methodology overview5

    Methodology Overview

    Define

    Measure

    Analyse

    Engineer

    Control

    Tollgate

    Tollgate

    Tollgate

    Tollgate

    Tollgate

    • Clarify Problem

    • Achieve consistency between:

      • Problem Statement

      • Business Case

      • Goals & Objectives

    • Obtain unbiased view of the requirements

    • Agree on project timeline

    • Develop macro view of process(es) involved

    • Establish baseline

    • Stratify problem or opportunity to a component level specific enough to analyse

    • Establish key areas of the process where the data are collected

    • Establish valid data collection plan

    • Remove or account for measurement variation

    • Finalise problem statement

    • Identify the critical factors driving the requirement(s) (Y’s)

    • Identify improvement impact

    • Find root cause(s) of variation

    • Generate solution

    • Develop and test improvements

    • Complete pilot testing

    • “Should Be” process

    • Develop cost/benefit

    • Develop/build implementation plan

    • Validate improvement

    • Establish new performance levels

    • Sustain good performance levels

    • Establish corrective & contingency action plan

    • Translate & transfer learnings

    • Celebrate!

    Steps:

    1.Complete Team Charter

    4. Map and Analysethe Process

    7. Identify Sourcesof Variation

    10. GenerateSolution Ideas

    13. Implement Solution

    2. Specify Customer

    Requirements & MGP

    5. Remove Measurement Variation & Collect Data

    8. Rank Key Causesof Variation

    11. Select Best Fit Solution

    14. Monitor Processand Results

    3. Complete High Level Process Map

    6. Determine Process Capability

    9. Validate Root Causes

    12. Test Solution and Confirm Results

    15. Replicate and Share Best Practice


    Module objectives3

    Module Objectives

    By end of this module you should be able to

    • Understand the fundamentals of ANOVA and statistical regression

    • Use these tools to confirm or deny the superiority of one process or group over others

    • Assess differences in performance between several groups or processes

    • Understand the nature of interaction between two X-Factors

    • Use regression to predict performance Y from the X-variable

    • Use JMP to conduct the analyses


    Identify and validate key x s

    Identify and Validate Key X’s

    • One-way and two-way ANOVA

    • Simple regression

    Region 1

    Region 2

    Region 3

    Region n

    Case 4


    Why do we need one way anova

    Why do we Need One-Way ANOVA

    • One-way ANOVA allows a comparison of performance characteristics across several groups

    • ANOVA determines if any of the groups give a significantly better performance

    • With one-way ANOVA we can identify solid benchmark performances

    • One-way ANOVA is a very robust statistical tool


    When to use one way anova

    When to Use One-Way ANOVA

    • When there are more than two samples

    • Or instead of Two Sample t Test

    • Whenever the means of three or more groups (regions, type of process, promotional campaigns) need to be compared

    • Whenever it is important to find a “best in class” group, if one exists


    Some practical questions for one way anova

    Some Practical Questions for One-Way ANOVA

    • Does the mean Cross Selling ratio differ by district or region? Which districts have a high and which have a low ratio?

    • Does the small business loan cycle time differ by districts? Which districts have the shortest mean cycle times, which the highest? Begin to look for reasons behind these performance differences

    • Does the profitability of futures trading customers depend on the maintenance level required by management? Maintenance levels are categorized as low, medium and high. Which one of those three levels is highest in profitability?


    One way anova design

    3

    3

    3

    2

    2

    1

    1

    1

    1

    2

    3

    2

    3

    2

    2

    3

    3

    1

    1

    1

    1

    2

    One-Way ANOVA Design

    Western region by districts: 22 branches sampled

    Group 1Southwestern

    8 Branches sampledMean 1

    Group 2Buffalo

    7 Branches sampledMean 2

    Group 3Niagara

    7 Branches sampledMean 3

    X Factor: Districts within Western region. Y Response: Cross-Selling Ratio

    Data are in Sam22WesternCrossSell.jmp


    Examine western region

    Examine “Western Region”

    • Need to identify “Best Performance” for Cross-Selling!

    • Western consists of three districts

      • Buffalo

      • Niagara

      • Southwestern

    • Are the Cross-Selling averages between these districts of the Western region the same, or are they significantly different from each other?

    • Which districts within that region perform(significantly) better or best?


    Hypotheses in anova

    Type of hypotheses in one-way ANOVA

    Null Hypothesis: All group means are the same

    Alternative hypothesis: One mean is different from the rest

    Alternative hypothesis: All three means are different from each other

    Sample Means

    Mean 1 = 2.81

    Mean 3 = 2.97

    Mean 2 = 3.37

    2.82.93.03.13.23.33.4

    Cross-selling ratio sample results from Samm22WesternCrossSell.jmp

    Hypotheses in ANOVA

    Which of these means are significantly different from each other?


    Identifying the source of variation overall versus group means

    Overall Mean explains the data well?

    OR should the groups means be used instead?

    Identifying the Source of Variation – Overall Versus Group Means?

    Where should we focus our analysis…overall mean or group means?


    Step in jmp to use one way anova 1

    Sam22WesternCrossSell.jmp.

    1. Use Fit Y by X

    2. Scatterplot with overall mean

    Step in JMP to use One-Way ANOVA (1)


    Step in jmp to use one way anova 2

    3

    3.1

    Select means/ANOVA

    p-value < 0.05 →Reject H0!

    But which means are different?

    3.2

    Step in JMP to Use One-Way ANOVA (2)

    Individual 95% confidence intervals are to wide to be conclusive


    Step in jmp to use one way anova 3

    4

    4.1

    Select compare means – each pair

    Which are different?

    Connected Letters Report

    4.2

    Significant difference? Check if CI includes 0!

    Two-sided

    Ordered Differences Report

    Step in JMP to Use One-Way ANOVA (3)

    Table of pairwise Means, 95% Confidence Intervals, & 2-sided p-values


    Summary of western districts using 22 branches

    LevelMean

    NIAGARAA 3.376

    SOUTHWESTERNAB2.975

    BUFFALO B2.814

    Levels not connected by same letter are significantly different.

    4

    3.5

    Total X-sell

    3

    2.5

    2

    BUFFALO

    NIAGARA

    SOUTHWESTERN

    District

    Summary of Western Districts Using 22 Branches

    • Niagara has a significantly higher Cross Selling ratio than Buffalo. The difference in average Cross-Selling between Niagara and Southwestern is significant at the one-sided, but not at the two-sided level

    • Niagara appears to be the district with the highest Cross Selling ratio, although one can be sure about the difference with Southwestern. A few additional samples should clarify this


    Pitfalls with one way anova

    Pitfalls With One-Way ANOVA

    • Observations in the groups have widely different variability. This can influence the p-values. Use “Unequal variances” from the options menu if that occurs

    • The sample sizes in some of the groups are inadequate to test differences with other groups. The only way out of this situation is to add more observations

    • There may be other ways to group the data. For example, instead of district, one could group branches by size, number of employees, etc. In that case it may be best to choose a higher-factor ANOVA, such as the two-factor ANOVA

    • The ANOVA discussed in this section is for means. This is called fixed effect ANOVA. There are applications where standard deviations are compared. Those are called random effects ANOVA. They are not treated here


    What to do using jmp in one way anova

    What to do Using JMP in One-Way ANOVA

    • Y variable is continuous

    • X Factor is a nominal variable with k ≥ 2 different levels

      • Factor could be label, like “Western” region

      • Factor could also be a numeric value, like region “1”. Be sure to turn it into nominal variable before using JMP

    • Use JMP platform Fit Y by X or Fit Model

      • ANOVA/means

      • Compare means

    • Output generated within:

      • Traditional ANOVA is overall test (not very useful!)

      • Dot plot by level and mean diamonds

      • Mean, Std Dev

      • F ratio with p-value to test equality of means

      • Confidence intervals for differences of means


    Check assumptions underlying anova

    Check Assumptions Underlying ANOVA

    • Independent observations both within and between groups

      • Make sure of this when data are collected

    • Group variances should have the same standard deviation

      • Test this in Fit Y by X using Unequal Variances submenu. Choose p-value from Levene test to decide whether or not group standard deviations are equal (YES when p>0.05, NO when p<0.05)


    Cross selling in western district levene test

    Sam22WesternCrossSell.jmp

    5

    6

    The Levene test is marginally not significant. So the results assuming equal variances apply.

    Factor: District

    Levels: Buffalo, Niagara, SouthWestern

    Small sample sizes make significant results difficult to achieve.

    The p-value (=0.0351) of the Welch ANOVA tests is close to the p-value assuming equal variances (p=0.0475). Both tests come up with the same conclusion. But the Welch tests is not needed here.

    Cross Selling in Western District: Levene Test


    Exercise stock price indexes by sector

    Exercise – Stock price Indexes by Sector

    Goal: Practice ANOVA skills

    Instructions:

    • 40 different stocks sampled, 10 per each of 4 sectors

      • Compare average performance of sectors

      • Characterise in detail which sectors have a different average performance from other sectors

      • Risk is often associated with variability of returns. Discuss whether these four sectors have significantly different risk

      • Look at the residuals to determine whether they follow a normal distributions as the assumptions often (but not always) require

    Time:Teams:15 minutes

    Report out:15 minutes


    Solution to stock index example 1

    Solution to Stock Index Example (1)

    • Observe the data plot with mean diamonds

      • Mean of manuf. group is highest (=1.183)

        • Is it significantly higher than the other means?

      • Mean of chemical group is lowest (=1.0125)

        • Is it significantly lower than the other means?

      • Variability within each group similar, so risk in each group is about the same

        • Using unequal variances – Levene test shows that there is no significant difference between industry group variables

    • Null Hypothesis: All means are equal can be rejected, because p-value of F-ratio in ANOVA table is 0.0071 which is below the a=0.05 threshold

    • Conclude that some group means are significantly different

      • Which means are significantly different? -> see next slide


    Solution to stock index example 2

    Solution to Stock Index Example (2)

    • Statistically significant differences in means

      • Manuf. mean is significantly different from chemical and service mean, but not from electronics mean

      • Chemical, electronics, and service are statistically not different

      • Electronics is in-between the two groups. More data might show it to belong to either the lower of higher group or to become a separate group

    • Risk is the same in each group, because variances (or Std Dev) of groups are statistically not different (Levene test not shown)

    • Normal quantile plot of residuals shows all within double flared region. Data seem to be normally distributed. All conclusions can be made without reservations about assumptions


    Summary5

    Summary

    • Used to compare several means from independent samples

    • Null Hypothesis: All (group) means are the same

    • Alternative hypothesis: At least one pair of group means differs

    • Overall test use “Prob>F” p-value to accept/reject H0

    • For identifying specific significant differences use compare means - “Each pair student’s t”

      • Prob >F p-value should be less than <0.05

    • Assumptions check

      • Group Std Dev are equal? Use UnEqual variances submenu. Use only Levene test, ignore others

    • Check normality using normal quantile plot


    Why do we need two way anova

    Why do we Need Two-Way ANOVA

    • Factors often interact creating synergistic or antagonistic relationships at factor level combinations. This is why two-way ANOVA is a necessary tool

    • Helps identify specific factor combinations that increase or decrease performance significantly

    • With two-way ANOVA ascertain the strength of influence of two X factors

    • Identifies how strong each factor contributes to the variation in the performance variable (CTQ)

    • Simultaneous comparison two factors results in more sensitive and efficient conclusions


    Some practical questions for two way anova

    Some Practical Questions for Two-Way ANOVA

    • How does the profitability of futures customers depend on required “maintenance level by management” (X1) and on whether or not customers fall into one of the two “proactive/reactive?” (X2) categories?Is there a combination of factors that is significantly more (or less) profitable than others?

    • Does customer satisfaction depend on the size of the branch where they conduct their banking and on “type of loan?”


    Factor level combinations in two way anova designs

    Factor B

    Level B1

    Level B2 Level B3

    Level A1

    A1,B1

    A1,B1

    A1,B2

    A1,B2

    A1,B3

    A1,B3

    Factor A

    A2,B1

    A2,B1

    A2,B2

    A2,B2

    A2,B3

    A2,B3

    Level A2

    Factor Level Combinations in Two-Way ANOVA Designs

    Factor B

    1

    One observation per factor level combination

    Level B1

    Level B2

    Level B3

    Level A1

    A1, B1

    A1, B2

    A1, B3

    Factor A

    A2, B1

    A2, B2

    A2, B3

    Level A2

    X Factor A

    2

    Two or more observations per factor level combination


    Profitability factors on individual futures trading accounts

    Profitability Factors on Individual Futures Trading Accounts

    Do levels of maintenance by management and reactive/proactive affect profitability of futures trading accounts?

    Profitability of customers in $10,000

    X2 Factor Maintenance by Management

    Medium

    High

    Low

    19, 11

    20, 17

    22, 31

    Reactive

    X1 Factor

    (Reactive and Proactive)

    27, 29

    25, 30

    31, 49

    Proactive

    12 randomly selected customers are classified into maintenance by management categories low, medium, and high depending on how much effort management has to spend on this customer. Customers are also classified by their behavior into a reactive and a proactive group


    Means for futures profitability example

    Means for Futures Profitability Example

    s

    n

    50

    ea

    M

    Proactive

    S

    40

    0 L

    0

    30

    Reactive

    00

    1

    $

    20

    n

    i

    t

    i

    f

    o

    r

    10

    P

    High

    Medium

    Low

    Maintenance by Management

    Profitability seems to increase as Maintenance levels decreases

    Proactive customers appear to be more profitable than Reactive customers

    Which of these differences are significant?


    Two interaction plots

    Two Interaction Plots

    Effects of reactive/proactive & maintenance on response=mean profitability

    Interaction

    No interaction

    Profitability

    Profitability

    $

    $

    Proactive

    Proactive

    Reactive

    Reactive

    High

    Medium

    Low

    High

    Medium

    Low


    Hypothesis in two way anova

    Hypothesis in Two-Way ANOVA

    • For each factor the Null Hypothesis assumes that all level means are the same

    • For two-way ANOVA with 2 or more observations per factor level combination there are three null-hypotheses

      • Factor a (reactive/proactive) means are all the same!

      • Factor b (maintenance by management) means are all the same!

      • There are no interaction effects between factors a and b!

    • The three matching alternative hypotheses state, that at least one of these means is different from the others

    • JMP uses effect test p-values in Fit Model to make decision


    Identifying sources of variation1

    SSError

    SSFactor A

    Identifying Sources of Variation

    Two-way ANOVA decomposes the total variation into four parts

    Total variation

    SSTotal

    Variation due to factor B alone

    Variation due to factor A alone

    SSFactor B

    Variation due to interaction of A & B

    Unexplained random variation (noise)

    SSInteraction AB


    Effect tests

    Effect Tests

    Effect tests p-values for the significance of factors and interaction

    LS Means plot from interaction window options gives a view of means of factor level combinations

    50

    Proactive

    40

    30

    Profit in $10,0000 LS Means

    Reactive

    20

    10

    High

    Medium

    Low

    Maintenance by management


    What is an interaction

    What is an Interaction?

    • Occurs when effects of one factor vary according to levels of other factor

    • When significant, interpretation of main effects (a & b) is complicated

    • Can be detected

      • In data table, pattern of cell means in one row differs from another row

      • In graph of cell means, lines cross


    Two way anova set up in jmp

    X2 Factor Maintenance by Management

    Medium

    High

    Low

    X Factor

    (Reactive and Proactive)

    19, 11

    20, 17

    22, 31

    Reactive

    27, 29

    25, 30

    31, 49

    Proactive

    Two-Way ANOVA Set-Up in JMP


    Two way anova uses fit model

    1

    2

    Two-Way ANOVA Uses Fit Model


    Terms from the fit model output in jmp

    Terms From the Fit Model Output in JMP

    • Effect tests for p-values of each factor including the interaction

      • Effect test window is shown in earlier slides

    • LS Means to have numerical values for means

      • LS stands for least squares – these are they only means to use

        These are options to be accessed at factor windows

    • LS Means plots to show means of factor levels and factor level combinations

    • LS Means student’s t to get confidence intervals for the differences between factor levels and between factor level combinations


    Least squares means

    From Effect Test: p=0.0197

    From Effect Test: p=0.0832

    From Effect Test: p=0.8688

    Least Squares Means

    For each factor ask: Which of these means are different from each other?

    LS Means are the better means. These are they only ones to use in ANOVA


    Ls means plots

    33.25

    23

    21.5

    31.83

    20

    40

    28

    27.5

    26.5

    18.5

    15

    LS Means numbers added from previous slide

    LS Means Plots


    Ls means differences for maintenance

    Crosstab Report for LSMeans Difference

    Which levels are different?

    Differences are

    High – mediumNS

    High – low sig.

    Medium – lowNS

    High-low

    Std err dif

    Lower 95% CL for dif

    Upper 95% CL for dif

    1

    How can one tell significance?

    1.

    2

    2.

    LSMeans

    Connected Letters

    Report

    LS Means Differences for Maintenance

    • Check if confidence intervals for differences include 0. If not, then the difference is significant.

      • High versus Low is sig. because CI is from -22.996 to -0.504.

    • Check letters on connected letters table. Levels with the same letter are not significantly different .

      • Difference High versus Low is significant, because these levels do not share the same letter.


    Interaction plot

    Interaction Plot

    The interaction plot gives two views of the same interaction.

    Sometimes one view is more instructive.

    Take your pick!

    Two views of same interaction!


    Futures customer profitability the actual example

    Futures Customer Profitability – The Actual Example

    Profitability = net after servicing and assigned

    Contingency table

    Reactive/proactive?

    Count

    P

    R

    High

    18

    12

    30

    Maintenance level per management

    Medium

    110

    90

    200

    Low

    22

    42

    64

    150

    144

    294

    Table contains frequencies of factor level combinations

    The real data file does not have an equal number of observations per factor/level combination. LS Means and regular means in the least squares tables do not agree!!! Only use LS Means


    Setting up for two way anova in jmp 1

    Data excerpt of first 8 observations

    Specify a 2-way model with interaction between main effects

    Setting up for Two-Way ANOVA in JMP (1)


    Setting up for two way anova in jmp 2

    Effect tests

    Source

    DF

    F Ratio

    Prob > F

    Maintenance level per management

    2

    8.4634

    0.0003

    Reactive/ proactive?

    1

    7.2719

    0.0074

    Maintenance level per management*reactive/ proactive?

    2

    4.2316

    0.0154

    p-value column

    Setting up for Two-Way ANOVA in JMP (2)

    Y=Net after service and assigned significance of X-terms

    All p-values less than 0.05. All effects are statistically significant


    Setting up for two way anova in jmp 3

    150000

    Net After Servicing

    and Assigned

    50000

    -50000

    High

    Medium

    Low

    Maintenance Level per Management

    LS Means plot of interaction shows results best

    150000

    Net After Servicing

    and Assigned

    50000

    Low

    Medium

    -50000

    High

    P

    R

    Reactive/ Proactive?

    Setting up for Two-Way ANOVA in JMP (3)

    150000

    Net After Servicing

    and Assigned

    50000

    -50000

    P

    R

    Reactive/ Proactive?


    Two factor anova interaction means and significance

    Connected Letters Report

    Two-Factor ANOVA – Interaction Means and Significance


    Two way anova interaction ordered differences report

    Statistically Significant

    Two-Way ANOVA Interaction – Ordered Differences Report


    Two factor anova interaction plot

    Two-Factor ANOVA – Interaction Plot

    200000

    150000

    Net after servicingand Assigned

    Maintenance Level

    per Management

    100000

    Maintenance Level

    per management

    50000

    Low

    0

    Medium

    High

    -50000

    P

    200000

    150000

    Net after servicingand assigned

    100000

    Reactive/Proactive?

    Reactive/Proactive?

    50000

    R

    0

    -50000

    P

    R

    Low

    High

    Medium


    Pitfalls of two way anova

    Pitfalls of Two-Way ANOVA

    • No data are available on one of the factor level combinations. The only easy solution is to eliminate one of the factor levels

    • Only one observation per factor level combination is available. This is OK, but one can not evaluate the presence of interactions

    • When data do not have the same number of observations for each factor level combinations, one must use the LS Means. Regular means are then inappropriate

    • Be aware the parameter estimates are difficult to interpret when the factor has three or more levels


    Exercise comparison of databases

    Exercise – Comparison of Databases

    Goal: Practice two-way ANOVA skills

    Instructions:

    • Four data bases compared with respect to speed of execution on 6 tasks

      • State the hypotheses concerning the comparison of the 4 database system

      • Would it have been Ok to ignore the factor Task (6 levels)? What would have been gained or lost by ignoring this factor?

      • Obtain results using JMP. The data is in databaseRBD.JMP

      • From the results above, is one data base system the clear winner? Very briefly justify your results

      • This analysis did not include an interaction term. Was this an omission of the analyst or is there another reason. Be brief!

    Time:Teams:15 minutes

    Report out:15 minutes


    Solution2

    Solution

    • Null Hypothesis of effect tests

      • Mean(I) = Mean(II) = Mean (III) = Mean(IV) for databases

      • Mean(1) = ••• = Mean (6) for tasks

    • Tasks factor must be included, because it contributes to variation. Leaving out Tasks factor leads to non-significant results for databases (p=0.95)

    • There is no clear winner! Mean(IV)=164 is not significantly different from Mean(II)=174. Thus these two database represent the best alternatives. Both means are significantly lower than Mean(I) and Mean(III)

    • No Interaction, because there is only one observation per Database*Task combination


    Solution3

    Solution

    • Only one combination of task and database type is available in DatabasRBD.jmp. This is the reason why no interaction can be included in the model. The interaction would be useful to show whether one database does particularly well or poorly in performing one or more task. It can’t be done, because the data are not there

    • In Two-way (and multi-way) ANOVA, to tests significance of factors select the Effect Tests output. For each Factor (including interaction when available) it will show a p-value relating to the Null Hypothesis that there is no difference. In the effect test below, both facts have very small p-values (<.0001). Thus there are significant differences in the mean time to perform various task. More importantly there are differences between the databases

    • Ordered differences shows that database II and IV are not significantly different (as labeled by letter C). These two require significantly less time on average to perform these tasks


    Exercise customer satisfaction by branch size and loan type

    Goal: Practice two-way ANOVA skills

    Instructions:

    Read the situation described in the Notes of this page

    Restate the problem in terms of statistical hypotheses concerning both factors Branch size and Loan type

    Fit the appropriate statistical model using JMP. The data is in BranchLoanSatisfac.JMP

    Does branch size matter with regard to customer satisfaction?

    Does loan type matter with regard to customer satisfaction?

    Is this a situation where interactions can play an important role? If yes, what do the data say about interaction?

    Is there a “Branch size” – “Loan” combination that is a clear(statistically significant) winner of the customer satisfaction derby?

    Exercise – Customer Satisfaction by Branch Size and Loan Type

    Time:Teams:15 minutes

    Report out:15 minutes


    Solution of customer satisfaction by branch size loan type 1

    Solution of Customer Satisfaction by Branch Size & Loan Type(1)

    BranchLoanSatisfac.jmp

    Obtain this LS Means plot as an Interaction Plot by clicking on the red triangle on Branch size* Loan type bar. Select LS Means Plot

    Lines are not parallel which may indicate interaction. Highest satisfaction is with small branches and secured loans Significant?


    Solution of customer satisfaction by branch size loan type 2

    Solution of Customer Satisfaction by Branch Size & Loan Type(2)

    • Null Hypothesis: Factor level means are all the same!

      • Branch effect: Mean(small)=Mean(medium)=Mean(high)

      • Loan effect: Mean(secured)=Mean(personal)

      • Interaction effect

        • Mean differences between branch sizes are not affected by Loan type

        • Mean difference between secured and personal loan is the same for all three branch sizes

    • Branch size is highly significant (p=0.0002). There are differences in average customer satisfaction between branch sizes

    • Loan type is highly significant: Secured loans have a significantly higher satisfaction score than personal loans (p=0.0056)

    • Interaction is significant (p=0.0285). As a result one can say with statistical significance that the lines are not parallel


    Solution of customer satisfaction by branch size loan type 3

    Solution of Customer Satisfaction by Branch Size & Loan Type(3)

    • Interaction is significant

      • Mean difference between secured and personal loans at small volume branches is 8.67 with the 95% CI from 3.81 to 13.52

      • Mean difference between secured and personal loans at high volume branches is 1.00 with the 95% CI from -3.85 to 5.85 including 0

    • Mean(Small, secured) = 80.67 is significantly higher than all other combinations. This is the winner!

    • Lowest satisfaction in high volume branches

    Level LeastSq Mean

    Small, secured A80.7

    Medium, securedB75.3

    Small, personal B C72.0

    Medium, personal C D70.0

    High, personal D67.0

    High, secured D60.0


    Two way anova in jmp

    Two-Way ANOVA in JMP

    • Three data columns required (Y, X1, X2)

    • Y variable is continuous, X-factors are nominal

      • Two X-Variable are Nominal each with at least 2 different values

    • Always use Fit Model

      • do not use fit Y by X


    Summary6

    Summary

    • Two data designs are typical

      • Two-way ANOVA with one observation per factor level combination does not allow an interaction effect in the model

      • Two-way ANOVA with equal replication for each factor level combinations allows interaction effect

    • Most important results are from effect tests table and LS Means comparisons

    • Interactions plot most useful explanation of the data, if interaction is significant


    Identify and validate key x s1

    Identify and Validate Key X’s

    • One-way and Two-way ANOVA

    • Simple Regression

    Case

    Effect of X

    Performance

    of Y (CTQ)


    Why do we need simple regression

    Why do we Need Simple Regression?

    • To see how the performance variable is influenced or related to an input variable

    • To ascertain how changes in the input variable affect the performance variable

    • To predict performance using input values

    • To understand the relationship between performance and input. Is it linear, curvilinear, S-shaped? Does it have saturation levels?


    Generic uses of simple regression analysis

    Generic uses of Simple Regression Analysis

    Three objectives in using regression

    1.Summarize Data

    2.Rate of Change

    3.Predict Y from X

    Y

    1

    Changein Y

    Line as data summary

    2

    Change in X

    Y0

    PredictY0 at X0

    3

    X0

    x

    x+1

    x

    Observations


    Overview of simple regression analysis

    Overview of Simple Regression Analysis

    • Often referred to as straight line fit, but curves can easily be accommodated

    • Generic applications of simple regression analysis

      • Obtain reasonable summary of data with “linear” relationship

      • Estimate the rate of change between Y and X

      • Predict Y-variable at specific values of X

    • Data requirements

      • Y is continuous

      • X is continuous

    • Useful output

      • Scatter plots (often used as an early check for correlation)

      • Regression equation, regression coefficients

      • Confidence and prediction intervals


    Applications in business

    Applications in Business

    Explain relationships between

    • Sales (Y) and advertising expenditures (X)

    • Housing starts (Y) and interest rates (X)

    • Expenditures on clothing (Y) and family income (X)

    • Savings rate (Y) and per capita income (X)

    • Cost of a product (Y) and number of units produced (X)

    • Cost of advertising per minute (Y) and number of viewers or readers (X)


    Example sales versus advertising

    Example – Sales Versus Advertising

    • A bank spends various amounts of money on advertising to increase loans to customers

    • Advertising dollars are said to increased the amount of loans being underwritten. Advertising values represent the X variable, because one is trying to predict loans from it

    • Loans are measured in terms of $ loans amounts written by all branches in an advertising district. Loans $ is the dependent or Y variable

    • Problem is to estimate the effectiveness of advertising dollars

      • By how much does each advertising $ increase loan $?

      • Is the relationship linear or curvilinear? ( This important for estimating marginal effectiveness of advertising $!)

    • Start with fitting a linear (regression) relationship as a first approximation!

    • Check that the model is adequate for decision making


    Change in loan versus change in advertising

    Change in Loan $ Versus Change in Advertising $

    Is advertising cost effective?

    Y

    1

    Slope b1

    Change in loanby 100000$

    2

    Loans writtenin 100000$

    X = Advertising $

    x

    x+1

    Observations

    Is advertising cost effective?


    Interpretation of slope and intercept

    Interpretation of Slope and Intercept

    Y

    Yhat = b0+b1(X+1)

    Slope b1 =est. average change in Y per unit change in X

    b1

    Yhat = b0+b1X

    Unit changein x

    b0

    X

    (0,0)

    x

    x+1

    Intercept b0 = estimated average of Y when X=0

    Often used as a mere fitting constant and not itself of interest


    Useful relationships

    Useful Relationships

    Y

    Y

    Saturation level

    Saturation

    S-shaped curve

    Limited growth

    Threshold

    X

    X

    Y

    Y

    Exponential decay

    Exponential growth

    X

    X


    Setting up simple regression in jmp 1

    1

    Setting up Simple Regression in JMP (1)

    • Data are in Sales$Advertising$Ex.jmp

    • Data are in pairs of (Y,X)=(Loans in 100000$, Advertising in 10000$)

    • Both Y and X must be declared “continuous” in the JMP window


    Setting up simple regression in jmp 2

    2

    3

    Setting up Simple Regression in JMP (2)

    Both Y, response and X, factor variable must be continuous!

    • The results is a scatter plot

    • The horizontal axis represents X

    • The vertical axis represents Y

    • Both axes are numeric and continuous

    • Next step is to fit a model: Overall mean, straight line, curved line (polynomial), etc


    Fit line fits a straight line

    Loans in 100000 $ = 13.120814 + 1.5574123 advertising in 10000$

    Intercept Slope

    4

    5

    Fit Line Fits a Straight Line


    Summary of fit

    Summary of Fit

    6

    Summary of fit (fit line)

    R square0.762

    Rsquare adj0.722

    Root mean square error0.438

    Mean of response16.981

    Observations 8

    Fit mean

    Mean16.981

    Std Dev [RMSE]0.830

    Std error0.293

    SSE4.820

    7

    Loans in 100,000$ = 13.120814 + 1.5574123 Advertising 10,000$


    Rsquare r 2

    Rsquare = R2

    • R2 is measures how well the regression line fits the data

    • R2 sometimes called the coefficient of determination

    • R2 measures the proportion of the total sum of squares of the Y (as a measure of variability) that has been explained by the regression model

    • 0 ≤ R2 ≤ 1

      • R2 = 0 means there is no linear relationship between Y and X

      • R2 = 1 means there is a perfect linear relationship between Y and X

      • Typically good values for R2 are larger than 0.5, but smaller values can be very significant when there are sufficient data

    • R2 = 0.76 indicates very good fit, though far from perfect


    Overall mean versus regression line

    Overall Mean line

    Regression line

    Fitted line: Linear relationship between Y and X

    Ybar = Mean of Y: No relationship

    Summary of Fit

    Fit Mean

    Mean of Response

    16.981

    16.981

    Mean

    Compare RMSE

    Root Mean Square Error

    0.438

    0.830

    Std Dev [RMSE]

    Overall Mean Versus Regression Line

    Which line fits the data better?


    Interpreting parameter estimates table

    Interpreting Parameter Estimates Table

    Parameter estimates table

    Term

    Estimate

    Std Error

    t Ratio

    Prob>|t|

    Intercept13.12080.89514.66<.0001

    Advertising in 10000$1.55740.3564.380.0047

    Term

    Estimate

    Std Error

    t Ratio

    Prob>|t|

    Intercept

    b0 = Yhat at X=0

    Std Dev. of b0

    b0/Std.Error

    p-value for b0

    Slope =Advertising in 10000$

    b1 = change in Y per change in X

    Std Dev. of b1

    b1/Std.Error

    p-value for b1

    Parameter Table gives the p-value of the slope. If Prob>|t| of slope <0.05, the regression equation is said to be significant. The slope is significantly different from 0.


    Predicting future performance

    Predicting Future Performance

    Predicted values – Yhat – average of Y at X0

    • Predicted values Yhat have two interpretations

      • Estimated average value of Y at X

      • Estimated (future) predicted value of Y at X

      • Save predicted in fit Y by X

    • Averages values are less uncertain than predicted values

    • Confidence interval on average

      • Confid curves fit

    • Prediction interval on future observation

      • Confid curves indiv

    Yhat in fit Y by X - Linear fit


    Yhat predicted values

    Yhat = Predicted Values

    Yhat in Fit Model

    Advertising in 10000 $

    Loans in 10000 S

    Predicted loans in 100000 $

    2.00

    2.82

    2.40

    3.13

    1.88

    2.64

    2.06

    2.90

    16.80

    17.00

    16.90

    18.00

    16.00

    17.10

    15.85

    18.20

    16.24

    17.51

    16.86

    17.996

    16.05

    17.23

    16.33

    17.64

    Estimated average Loan $ for 20K $ Advertising:

    X=2.0, Y=16.8,→predicted value Yhat=16.24.

    Yhat = Predicted Loans $ are all on the regression line


    Pitfalls in simple regression

    Pitfalls in Simple Regression

    • Rsquare is influenced by a few outlier observations

    • Rsquare suffers because the data are not observed in a proper range of X-values

    • Extrapolation beyond the range of the data


    Illustration of pitfalls r 2 and outliers

    Illustration of Pitfalls - R2 and Outliers

    7

    7

    6

    6

    R2 = 0.68

    R2 = 0

    5

    5

    4

    4

    Y5

    Y6

    Y

    Y

    3

    Outlier

    3

    2

    2

    1

    1

    0

    0

    -1

    -1

    0

    1

    2

    3

    4

    5

    6

    0

    1

    2

    3

    4

    5

    6

    X

    X


    Illustration of pitfalls r 2 and range of x values

    7

    7

    6

    6

    5

    5

    4

    4

    3

    Y

    3

    Y

    2

    2

    1

    1

    R2 = 0.75

    R2 = 0.5

    0

    0

    -1

    -1

    0

    1

    2

    3

    4

    5

    6

    0

    1

    2

    3

    4

    5

    6

    X

    X

    Range of X from 2 to 4, with a mediocre R2

    Range of X from 1 to 5, with much stronger R2

    Illustration of Pitfalls - R2 and Range of x-values

    X-values need to be spread over reasonable range


    Illustration of pitfalls extrapolation

    Y

    Danger!

    Interpolation is OK

    Extrapolation

    Extrapolation

    Danger!

    Range of observed X values

    X

    Illustration of Pitfalls – Extrapolation


    Example relationship between customer files processed and number rejected

    Why do customers go over their credit limit?

    Is there a fixed set of customers that exceed the limit?

    Is there a systemic problem that encourages exceedances?

    Are the number of rejected files depending on the volume?

    Example – Relationship Between Customer Files Processed and Number Rejected

    ScatterACh.jmp


    Scatter plot with regression line

    Scatter Plot with Regression Line

    • The data indicate that the rejections are volume dependent

    • However the rate at which files are rejected is very high and systemic changes should be looked into

    600

    550

    500

    Jun-02

    Total rejects

    450

    400

    350

    3500

    4000

    4500

    5000

    Customer files processed

    Total Rejects = -103.3791 + 0.1331215 Customer files processed

    Intercept Slope


    Regression example summary

    Regression Example Summary

    • Regression analysis is a good method when applied to the Loan $ versus Advertising $ Example

    • Slope coefficient is significantly different from 0

      • Advertising $ affect Loan $ positively -> one-sided HA, cut p-value in half

    • RMSE is much smaller the Std Dev around mean, so regression explains a lot of the variability in Loan $

      • Linear Regression is a good summary of the data

    • R2 likewise is very high and indicates a good fit

    • The Model assumptions – constant variance, normal distribution of residuals – seem to be met

      • This regression can be used for predicting Loan $ from Advertising

    • More observations would be nice and add credibility to these very promising results


    Key simple regression steps

    Key Simple Regression Steps

    • Determine what needs to be answered

      • Estimate the relationship between Y and X

      • Estimate average performance (confidence interval)

      • Predict future performance

    • Fit regression equation by estimating its slope and intercept

      • Regression line is called “Model”

      • Regression coefficient are slope and intercept

    • Test the slope for its significance using the p-value of the associated t-Ratio

    • Determine how well the data fit the simple regression model by examining

      • Root Mean Square Error

      • Rsquare

      • Plot of the data with the fitted line

    • Obtain and summarize all important output

    • Interpret results and communicate them


    Summary of simple regression analysis in jmp

    Summary of Simple Regression Analysis in JMP

    • Y - Variable is continuous

    • X - Variable is continuous

    • JMP platform: Fit Y by X (or Fit Model)

    • Input: single column of Ys, single column of Xs

    • Output generated within

      • Slope and intercept of equation

      • t-Test with p-value for slope an intercept

      • RMSE, Overall Mean of Y, RSquare

      • Confidence and Prediction Intervals (not covered)


    Exercise housing starts versus interest rates

    Exercise – Housing Starts Versus Interest Rates

    Goal: Practice regression skills

    Instructions:

    • Is the fit of interest rates and housing starts either good or poor?

      • Explain the slope coefficient b1 = -26.13046. What is the p-value? Does it support a “significant regression line”?

      • Estimate the number of housing starts for an interest rate of 8% and find a 95% confidence interval for the number of housing starts for that interest rate

      • You need to predict housing starts for the coming month for which the interest rate is almost certainly going to be 6.75%. What do you need to do?

    Time:Teams:15 minutes

    Report out:15 minutes


    Solution4

    Solution

    • This is a relatively good fit

      • R2 = 0.44

      • RMSE=21.3, reduced from StdDev(Y)=27.2 without the X-variable

      • Plot indicates at most one poof fitting observation (see arrow)

    • Slope b1=-26.1 estimates that on average, for every increase in interest rates of 1%, housing starts go down by 26.1

    • P-value of slope = 0.0181 is used to reject the Null Hypothesis that slope is equal to 0. In this example the regression slope is significantly difference from 0

    • Average housing starts when interest rate is 8% (Yhat at X=8) is estimated as 216.43

    • Predicted housing starts when interest rate is 6.5% (Yhat at X=6.5) is estimated as 249.09


    Summary7

    Summary

    • Correlation does not imply causation

    • Slope measure average rate of change in Y per change in X

    • Check for significant rate of change using the p-value of slope coefficient

    • Predicted values can be used to predict future performance (with caution – see pitfalls)

    • Check for good fit using R-Square and reduction in standard deviation (RMSE)

    • Pitfalls need to be avoided. They can ruin a good analysis


    Analyse phase6

    Analyse Phase

    Module 1

    Recap of the Measure Phase

    Module 2

    Overview of the Analyse Phase

    Module 3

    Graphical Data Analysis

    Module 4

    Simple – Identify, Rank and Validate Key X’s

    - 5 Why

    - Cause and effect diagram

    - Multi-voting

    Module 5

    Validate the Vital Few

    - One sample methods

    - Two sample methods

    - Chi-Square

    Module 6

    More Advanced – Identify, Rank and Validate Key X’s

    - ANOVA

    - Simple regression

    Advanced – Identify, Rank and Validate Key X’s

    - Introduction to design of experiments

    Module 7

    Module 8

    Tollgate


    Methodology overview6

    Methodology Overview

    Define

    Measure

    Analyse

    Engineer

    Control

    Tollgate

    Tollgate

    Tollgate

    Tollgate

    Tollgate

    • Clarify problem

    • Achieve consistency between

      • Problem statement

      • Business case

      • Goals & objectives

    • Obtain unbiased view of the requirements

    • Agree on project timeline

    • Develop macro view of process(es) involved

    • Establish baseline

    • Stratify problem or opportunity to a component level specific enough to analyse

    • Establish key areas of the process where the data are collected

    • Establish valid data collection plan

    • Remove or account for measurement variation

    • Finalise problem statement

    • Identify the critical factors driving the requirement(s) (Y’s)

    • Identify improvement impact

    • Find root cause(s) of variation

    • Generate solution

    • Develop and test improvements

    • Complete pilot testing

    • “Should be” process

    • Develop cost/benefit

    • Develop/build implementation plan

    • Validate improvement

    • Establish new performance levels

    • Sustain good performance levels

    • Establish corrective & contingency action plan

    • Translate & transfer learnings

    • Celebrate!

    Steps:

    1.Complete Team Charter

    4. Map and Analysethe Process

    7. Identify Sourcesof Variation

    10. GenerateSolution Ideas

    13. Implement Solution

    2. Specify Customer

    Requirements & MGP

    5. Remove Measurement Variation & Collect Data

    8. Rank Key Causesof Variation

    11. Select Best Fit Solution

    14. Monitor Processand Results

    3. Complete High Level Process Map

    6. Determine Process Capability

    9. Validate Root Causes

    12. Test Solution and Confirm Results

    15. Replicate and Share Best Practice


    Module objectives4

    Module Objectives

    By end of this module you should be able to

    • Understand the uses and benefits of DOE in the financial services industry and in IT

    • Understand the DOE Factorial and Screening Designs

    • Use DOE to solve a process/product problem or an IT (technology) development or implementation

    • Generate common designs using JMP

    • Analyse the results of experimental data


    Present situation

    Present Situation

    • Efforts usually focused on leveraging DMAEC

      • Aimed at continuous improvement

      • Requires tactical deployment

    • Need exists for a technique for Financial Services BB’s to

      • Prevent performance/process issues rather than fixing them

      • Implement improvement changes before product or services goes into use

      • Have a design solution rather than day-to-day process improvement solutions

    • Design method, using DOE is being developed to meet precisely that need


    Why do we need doe

    Why do we Need DOE?

    • DOE allows examination of many factors in a systematic and efficient way

    • DOE helps identify those X’s (factors) that have the biggest influence on performance

    • DOE can be used in pilot or roll-out projects to fine tune the system for best performance

    • DOE can be used with simulation to gain understanding without having to run the actual system


    Why do we need doe in designing new products or processes

    Why do we Need DOE in Designing New Products or Processes?

    • Lower cost and higher quality are achieved by developing a robust product or service

    • Robust means the product/service is less sensitive to changes in materials, processes, operators, applications, and environments

    • No matter what product/process changes occur, the service performance meets the target

      The results?

      1. Can use low-cost options

      2. Process costs are reduced/risk mitigated; less control required

      3. Quality performance improves


    Doe as a major platform

    DOE as a Major Platform

    D

    M

    A

    E

    C

    Business

    problem

    Practical problem

    Quantifyproblem &

    identify gaps

    Statistical problem

    DOE

    Identify improvement levers

    Statistical solution

    Implement solutions

    Practical solution

    DOE can be leveraged to deliver the DMAE phases solutions!


    Proposed execution template

    Proposed Execution Template

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control

    Depending on strategy

    • Identify vital few factors

    • Estimate factor effects

    • Estimate optimum factor combination & setting

    (1) Identify problem

    • Complete charter

    • Problem statement

    • Opportunity

    • Identify factors (X’s)

      • C-E Diagram

    • Decision strategy

      • Screening design

      • Full factorial design

      • Response Surface design

    • Plan design

      • Select design

      • Modify design

    • Define experimental region

      • Set factor levels

      • Select design

    • Run experiment

    • Implement

      • Implementation Plan

    • Pilot solution

      • Gantt chart

      • Stakeholder analysis

      • Communications plan

    • FMEA

    • BPRM

      • Control chart

      • SOP (FIM)

    Basic quality tools are used to set the design

    JPM increases the ease of application

    Tollgate

    Tollgate

    Tollgate

    Tollgate

    Tollgate


    Providing reference for the experimental terminology

    Providing Reference for the Experimental Terminology

    Terms

    • We will leverage the construct of the causal system as a reference

    • The effect is called the “ Y response,” and the cause is called a “X factor.”

    • Factor levels (X’s) determine the Y response or performance

      Our Challenge

      a. Determine (Y) – - response

      b. Identify & set factors

      c. Select & adjust design

      d. Run & analyse experiment

    Framework/reference

    1

    Cause

    Effect

    2

    X factor

    Y response

    Response functionf (factors)

    Y

    Response 2

    3

    Response 1

    X factor

    Level 1

    Level 2


    Basic terminology

    Basic Terminology

    • Factors or X variables

      • Variable be manipulated by the experimenter

      • Quantitative factor: size of staff

      • Qualitative factor: Digitized versus paper

    • Response, performance or Y variables

      • Quantitative Y: Loan approval time, counts: defects per unit

      • Qualitative Y: ratings of poor – good – excellent

      • Binary (0-1) Y: success – failure

    • Factor effect

    • Experimental run

      • Operation of the process at fixed factor levels

      • Results in at least one performance measurement

      • Factor levels between experimental runs are varied according to strictprotocols


    Using design strategy to meet our goals

    Using Design Strategy to Meet our Goals

    Y

    X1,X2,…,Xn

    Execution challenge

    Design strategy

    • Which input X-factors are important in determining the level of the Y performance variable?

      • Screening Designs

    • What combination of X-factors provide good performance of the Y variable?

      • Full Factorial Designs

    • Which level combination of X-factors give an optimum performance of the Y variable?

      • Response Surface Designs

    What are the important factors?

    What is the best way to run or execute?

    What is

    optimal way to run or execute?


    Uses and benefits of experimental design

    Uses

    Find important X factors affecting Y

    Estimate factor effects & interactions

    Find optimal factor combinations

    Benefits

    Efficiency of the data collection

    DOE provides more bang for the buck in terms of information

    DOE determines the necessary number of runs to obtain the desired information

    Results are easy to understand

    Designed experiments provide simpler and more meaningful interpretation of results than non-designed data collection methods

    Results can be extended

    Experimental designs can be extended and augmented to improve the clarify and further improve the understanding of the problem.

    Uses and Benefits of Experimental Design?


    Example loan processing times

    Example – Loan Processing Times

    • Mean loan processing times are too long. Some loans are quickly, but some take a long time and they contribute to highly variable loan processing times. It is decided to get at the bottom of this – But how?

    • 6 factors identified

      • Paper or digitized form of application

      • Staff level (10, 20)

      • Credit check (normal, rapid)

      • Application is complete? (Yes or No)

      • Processing center (old, new)

      • Loan amount (low, hi) with low anything below $1 million

    • DOE can help in evaluating factor effect with minimal data


    Two quick answers provided by doe

    Plot of scaled factor effects

    Pareto plot of factor effects

    Two Quick Answers Provided by DOE

    Data in ProcessTime6Fac.jmp

    Which factors have the largest effect

    on loan processing times?

    1. Staff Level seems to have high impact on processing time. High staff levels reduce processing time.

    2. Processing Center seems tied with Staff Level for most important factor. The New Processing Center takes considerably less time than the Old.

    3. Application Complete appears less important than 1 & 2. But complete Applications take less time than incomplete ones.

    Other factors appear not important!


    What is an experimental design

    $ $

    What is an Experimental Design?

    • Systematically varying one or more factors to measure the effect on a response variable under given experimental constraints

    • DOE may forcibly change factor settings of an existing system to understand underlying cause and effect relationships and to gain breakthrough knowledge

    • Factor combinations allow the experimenter to measure the factor effects more precisely and accurately

    • By nature DOE is iterative

    DOE focuses on efficient and systematic collection of data involving many factors

    DOE provides unique and exclusive insights into complex problems


    Doe for financial services roadmap

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Doe in define

    DOE in Define

    Model for defining the problem

    Leverage problem statement F(Y) = X1..Xn

    • On what object is the defect observed?

    • What exactly is wrong (defective)?

    • Where was the object with the defect first observed (geographically)?

    • Where on the object does the defect appear?

    • When was the defect first observed (clock/calendar/time)?

    • When in the life cycle of the object was the defect first observed?

    • In what pattern is the defect observed?

    • How much of the object is defective?

    • How many objects are defective?

    • What is the trend?

    Identity

    Location

    Timing

    Magnitude

    P

    To develop CTQ’S and later factors & levels

    From Plunkett and Hale, The Proactive Manager. J. Wiley and Sons, 1982)

    The problem statement identifies the response variable Y


    Doe for financial services roadmap1

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Investigating common cause variation

    Investigating Common Cause Variation

    Experiment


    Doe can provide breakthroughs in variation reduction

    DOE can Provide Breakthroughs in Variation Reduction

    • Critical outputs are variable

      • Because the process itself causes variability

        • Work habit differ between branches

      • Because some controllable inputs are subject to variation

        • Staffing levels are adjusted according to demand

        • Differences in promotional material used at different times

      • Because some inputs are variable and can not be controlled

        • Interest rates vary over time

    Uncontrollable inputs subject

    to variation

    Process causes variability

    Controllable inputs subject

    to variation

    Variability in output


    Design versus noise factors

    Design factors

    Any product or process/service design parameter that can be controlled

    Controllable

    Low-cost alternatives

    Reduce sensitivity to noise factors

    Typically nominal values

    Control factors affect mean and variability or variability only

    Adjustment factors affect the mean only

    Cost reduction factors have little effect on mean or variability

    Noise factors

    Inherent processing/operating variation, unit-to-unit differences under “same” conditions

    Uncontrollable

    Costly/undesirable to control

    Typically tolerances, variations different customer applications

    Deterioration of system components (internal noise)

    Variation in operating environment e.g. regulatory requirements (external noise)

    Human errors

    Design Versus Noise Factors


    Doe for financial services roadmap2

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Approaches to factor selection

    Approaches to Factor Selection

    • Brainstorming is an unstructured techniques to generate a list of possible factors

      • Multi-voting is used to prioritize the factors

    • Cause-effect diagram (Fishbone Diagram) is a more structured approach to generate factors

      • Multi-voting is used to prioritize the factors

    The cause and effect diagram is the method of choice in factor selection


    C e for identifying the factors

    C-E for Identifying the Factors

    When used

    • When necessary to identify all possible factors for an experiment, or when identifying relationships for potential process optimisation

      Explanation

    • The C-E analysis is an organized approach to representing the relationships between a “response” and all of its possible “factors”

    .

    Note:The C-E analysis can be (1) a guide for discussion when identifying factors to prevent participants from straying, and (2) a further reference if the experiment does not resolve the problem. The previous diagram may be used as a starting point

    The material on C-E is repeated from Module 4 to show its use in the DOE process


    Using c e for identifying selecting factors

    Using C-E for Identifying & Selecting Factors

    • General rules

      • Capture as many factors as required

      • Prioritize using the multi-vote

      • Subject matter expert should be encouraged to think beyond certain accepted practices or protocol

    Methods

    Design

    Factor

    Factor

    Factor

    Factor

    Factor

    Factor

    Response

    Factor

    Factor

    Factor

    Factor

    Factor

    Factor

    Materials

    Machine


    Example telephone survey

    Example – Telephone Survey

    Data collection

    Interviewer

    Sounds untrustworthy

    on phone

    Voices

    own

    opinion

    Untimely analysis

    Missing data

    Poor training

    Successful

    telephone

    survey

    Confusing

    question

    sequence

    Customer

    refuses answers

    Customer

    not at home

    Ambiguous

    questions

    Customer unable

    to respond

    Non-actionable

    questions

    Interview timing

    Questionnaire

    The material on C-E is repeated from Module 4 to show its use in the DOE process


    Example direct mail survey

    Example – Direct Mail Survey

    Customer

    Data

    Non-response

    Untimely

    analysis

    Wrong address

    Illegible response

    Questionnaire

    lost in mail

    Incomplete

    response

    Successful

    direct mail

    survey

    Sequence

    Length

    Non-actionable

    Ambiguous

    Layout

    Questions

    Questionnaire

    The material on C-E is repeated from Module 4 to show its use in the DOE process


    Doe for financial services roadmap3

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Setting factor levels

    Setting Factor Levels

    • How to set levels of continuous factors

      • Factor levels can be set in controllable factors

      • Factor levels should be close the reasonable operating boundaries

      • Factors at 2 levels are preferably set as high and low as possible

        • Example: Staff level should not be below 10 or higher than 20. Set Staff at 10 and 20

    • How to select categorical factors

      • Factor levels are distinct categories (old and new call center)

      • Factor levels can not be easily modified

      • Select as many levels (categories) as problem requires

    Make sure that factor level combinations do not result in operational difficulties!


    Region of experimentation factors

    Factor B

    Non-feasible

    region

    potential

    high

    feasible

    region

    actual

    high

    region of

    actual

    experiment

    Non-feasible

    region

    actual

    low

    potential

    low

    potential

    low

    actual

    low

    actual

    high

    potential high

    Factor A

    Region of Experimentation & Factors

    • Factor levels are carefully selected

      • To cover the region of interest (only staff from 10 to 20)

      • To be combined with other factor levels for maximum efficiency


    Doe for financial services roadmap4

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Recommended designs

    Recommended Designs

    Execution challenge

    Recommended design types

    • Which input X-factors are important in determining the level of the Y performance variable?

      • Screening Designs

    • What combination of X-factors provide good performance of the Y variable?

      • Full Factorial Designs

    • Which level combination of X-factors give an optimum performance of the Y variable?

      • Response Surface Designs

    What are the important factors?

    1

    What is the best way to run or execute?

    2

    What is

    optimal way to run or execute?

    3

    Note: see MBB for design adjustments


    Overview of factorial designs

    Overview of Factorial Designs

    • Full Factorial Designs

      • All possible treatment combinations of at least two factors, each run at two or more levels, are studied. The purpose is to estimate main effect and interaction effects

    • Fractional factorial experiment

      • A carefully chosen subset of all possible treatment combinations is selected. The purpose is to study Main Effects and possibly some, but not all interaction, effects

    • Response Surface design

      • Often an augmented experiment so that (1) the Y response can to be approximated by possibly curvilinear functions of the important X-factors or (2) the optimum setting of the X-factors needs to be determined

    Fractional factorial

    Full factorial

    Response Surface D.


    Factorial designs at two levels

    Factorial Designs at Two Levels

    • Most widely used design type

    • Each factor is evaluated at only two levels

    • Full Factorial Designs at 2 levels

      • Each level of a factor is combined with all possible combinations of all other factors

    • Fractional Factorial Designs 2 levels

      • Does not combine all possible factor level combinations

      • Factor level combinations are selected according to strict rules

    • Factorial Designs at two levels are used to estimatelinear relationships between Y response and X factors

    Factorial Designs at two levels systematically combine factors at two level combinations to give desired data for identifying important factors, and estimating the effect of those factors on the Y response variable


    2 factor design at two levels

    2 Factor Design at Two Levels

    • X factors have factor levels. These are combined with factor levels of other factors

    • Factor levels are combined for factorial design

      • Telephone contact(No or Yes – 2 levels)

      • Direct mail contact(No or Yes – 2 levels)

    • 4 possible factor level combinations

    Telephone +

    direct mail

    Telephone

    contact

    Factor telephonecontact(No, Yes)

    Factor direct mail(No, Yes)

    No

    contact

    Direct

    mail


    Reasons for factorial designs

    Reasons for Factorial Designs

    • Observe the effect of each controllable factor at more than one factor level

    • All factor levels are combined systematically

      • Precision of obtained results in estimating effects

      • Accuracy in covering Main Effects and interactions

    • Sub-experiments (blocks) from building blocks for future experimentation

    • Data structure yields easily interpretable data


    Uses and benefits of factorial designs at 2 levels

    Uses

    Allow identification of important factors

    Allow estimation of how large each factor effect is

    Some allow estimation of interaction terms

    Benefits

    They are very efficient for discovery

    They are sufficient for evaluating linear Response Surfaces

    They require the fewest number of runs per factor and overall

    They can be used as building blocks in sequences of experiments

    Uses and Benefits of Factorial Designs at 2 Levels

    Factorial Designs at two levels evaluate each factor at only two levels


    Factor effect in factorials at two levels

    Factor Effect in Factorials at Two Levels

    • Main effect of A

      • Average difference in Y as level of factor A changes by one experimental unit (from low to high)

    • Interaction effect A*B

      • Half the estimated difference in Y between Main Effects of factor A at the two levels of Factor B (and vice versa)

    • Interaction effect A*B*C

      • Half the estimated difference in Y between the 2-factor effect A*B at the two levels of Factor C (all combinations of A,B,C)

    • Minimum significant factor effect

      • smallest effect considered a significant result


    Interaction effects

    Interaction Effects

    • An statistical interaction effect between two or more factors measures the extent to which the effect of one factor depends on the levels of the other factor or factors

    No interaction

    Interaction

    Factor B

    Factor B

    Effect

    Effect

    Effect

    Effect

    Level 1

    Level 2

    Level 1

    Level 2

    Factor A

    Factor A


    Creating a full factorial design in jmp

    Creating a Full Factorial Design in JMP

    DOE platform in JMP

    • Very powerful

    • State of the art tool

    • Simplifying the creation of experimental designs

    2

    1


    A simple introduction to factorial designs

    Run

    Direct Mail

    Telephone

    Account Bal K$

    1

    No (-)

    No (-)

    10

    2

    No (-)

    Yes (+)

    12

    3

    Yes (+)

    No (-)

    14

    4

    Yes (+)

    Yes (+)

    16

    A Simple Introduction to Factorial Designs

    • New bank customers are selected into an “On Board” program, receiving either special direct mailings or telephone contacts or none of the above

    • A program to evaluate the effectiveness of such efforts assigns an equal number of customers to the contact categories

      • The response variable (Y) is account balance after 90 days in 1000$

      • Factor A: Direct mail contacts “Yes” or “No”

      • Factor B: Telephone contacts “Yes” or “No”


    A two factor experiment at two levels

    Direct mail

    Telephone

    Account bal

    Mail (No)

    Tele (No)

    10

    Mail (No)

    Tele (Yes)

    12

    Mail (Yes)

    Tele (No)

    14

    Mail (Yes)

    Tele (Yes)

    16

    A Two Factor Experiment at Two Levels

    • Full effect of factors mail and telephone

      • Direct mail: Mail(Yes) – mail(No) = [(14+16) – (10+12)]/2 = 4

      • Telephone: Tele(Yes) – tele(No) = [(12+16) – (10+14)]/2 = 2

    • Half effect of factors mail and telephone

      • Direct mail: (full effect)/2 = 4/2 = 2

      • Telephone: (full effect)/2 = 2/2 = 1

    Yes

    12

    16

    Telephone

    No

    10

    14

    Yes

    No

    Direct mail


    Doe for financial services roadmap5

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Creating a two factor design each at two levels

    Two factors at two levels:

    Factor A: Direct mail contacts

    Factor B: Telephone contacts

    1

    2

    Click and overtype default labels

    3

    • Other run orders selected by toggling

    • Keep the same

    • Sort left to right

    • Randomize

    • Sort right to left

    Creating a Two-Factor Design each at Two Levels


    Order determines order of rows in design table

    3

    To

    Sort left to right

    The levels of the first factor column varies slowest, the second varies faster, the third (if present) varies even faster

    Sort right to left

    The levels of the first factor column varies fastest, the second varies less fast, the third (if present) varies even slow

    Randomize

    The runs are presented in the random order in which the experiment should be performed

    Y

    Order Determines Order of Rows in Design Table


    Summary of steps for on board in jmp 1

    1

    2

    Click to create new data table with experimental conditions

    Summary of Steps for “On Board” in JMP (1)

    • List factors and response

      • Response: Acc ball in 10K$

      • Factor A: Mail contact – (Yes or No)

      • Factor B: Telephone contact– (Yes or No)

    • Specify run order to generated design

      • Sort left to right (for display only)

      • No center points or replicates

      • Centre points are part of Response Surface Designs

    1

    2


    Summary of steps for on board in jmp 2

    3

    Y

    Run experiment and enter the Y-response (Acc Bal)

    4

    Y

    Summary of Steps for “On Board” in JMP (2)

    Data table for 2 x 2 factorial “On Board” design


    Doe for financial services roadmap6

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Good practices in running an experiment 1

    Good Practices in Running an Experiment (1)

    • Run all required factor level combinations. Missing a design condition can nullify the results

      • At start, make sure all factor level combination make sense

      • At start, make sure sufficient resources are available to run all factor level combinations

      • At start, make sure that none of the conditions create hardships or are destructive

    • Make sure that the Y measurements are precise and without bias

      • DOE minimizes the number of runs required, the need to reduceor minimize measurement error is critical

    Get all required data


    Good practices in running an experiment 2

    Good Practices in Running an Experiment (2)

    • Randomize the run order as provided by JMP-DOE when generating the design matrix

    • Random order is also important with replication of the design

    • Random order may not be possible, because factor levels can not be changed easily changed

      • Use the actual run order in the analysis

    • Make sure that all experimental runs are conducted under similar conditions

      • Differences in personnel, environmental conditions, and other factors contribute to the outcome

      • Keep track of these potential influences and add them to the data matrixas separate variables to be used in case

      • Experimental runs conducted at different times or location should beespecially noted. In DOE – JMP, “Blocking” is designed to take care ofsuch problems. Note: See your MBB for help

    Randomise


    Good practices in running and experiment 3

    Good Practices in Running and Experiment (3)

    • In confirmatory DOE (full factorial) consider replicates runs

      • Replicates are the same factor level combination run several times

      • Replicates estimate the average response at a factor level combination more precisely

      • Measurement error can be reduced with replications

    • When conducting interview or survey types DOE, it is imperative to select as many as possible replicate respondents:

      • Observational studies often have less precise scales (see Questionnaires)

      • Cost of interviewing respondent is cheap relative to the cost of developing the protocol

    • In simulation replicates are free and are part of the process

    • Consider pilot runs

      • Make sure that the results are not influenced by your increasing skill in observingthe results

    Replicates


    Final comments pitfalls run order

    Order of experiment

    Staff

    Center

    Application complete?

    Time

    3

    4

    1

    7

    6

    8

    5

    2

    10

    10

    10

    10

    20

    20

    20

    20

    Old

    Old

    New

    New

    Old

    Old

    New

    New

    Yes

    No

    Yes

    No

    Yes

    No

    Yes

    No

    20.72

    28.89

    12.34

    12.53

    11.35

    18.49

    4.19

    6.78

    Final Comments – Pitfalls & Run Order

    Complete matrix:Eliminating one run leads to poor results, eliminating two runs leads to no results

    Run Order:The first condition is staff=10, new center, applications complete

    Easily done!


    Doe for financial services roadmap7

    DOE for Financial Services Roadmap

    • Define

      • Problem statement

      • Identify response variable Y

    • Set factors

      • Identify factors

      • Set experimental region

      • Select/adjust design

      • Set DOE in JMP

      • Run experiment

    • Analyse and interpret results

    DOE for Financial Services

    Define

    Set factors & run exper.

    Analyse

    Engineer

    Control


    Detail analysis steps for 2 2 factorial design 1

    1

    • Default model includes the two main effects gasoline and speed and the interaction effect gasoline*speed

    • Three model terms plus one overall mean equals the number of observations (4). No p-values, because of perfect fit

    3

    2

    or

    Detail Analysis Steps for 2*2 Factorial Design (1)


    Detail analysis steps for 2 2 factorial design 2

    4

    5

    6

    Data Table

    Detail Analysis Steps for 2*2 Factorial Design (2)


    Detail analysis steps for 2 2 factorial design 3

    Pareto plot of estimates

    Term

    Estimate

    Direct Mail[Mail(No)]

    -2.000000

    Telephone[Tele(No)]

    -1.000000

    Direct Mail[Mail(No)]*Telephone[Tele(No)]

    0.000000

    Detail Analysis Steps for 2*2 Factorial Design (3)

    Pareto plot of main effects and interactions

    7


    Detail analysis steps for 2 2 factorial design 4

    16

    Acc Bal

    10

    10

    Tele(Yes)

    Mail(yes)

    Tele(No)

    Mail(No)

    Direct Mail

    Telephone

    16

    Acc Bal

    14

    10

    Mail (No)

    Tele (No)

    Direct mail

    Mail (yes)

    Telephone

    Tele (Yes)

    Detail Analysis Steps for 2*2 Factorial Design (4)

    Prediction profiler

    Right panel shows setting of prediction profile plot for Mail (Yes) and Tele (No). The value in the circle (added here) is 14 and represents the average value for this setting

    8

    Left panel shows setting of prediction profile plot for Mail (No) and Tele (No). The value in the circle (added here) is 10 and represents the average value for this setting


    Detail analysis steps for 2 2 factorial design 5

    9

    LS Means plots

    18

    16

    14

    Acc Bal LS Means

    12

    10

    8

    Mail (No)

    Mail (yes)

    Direct Mail

    18

    16

    14

    Acc Bal LS Means

    12

    10

    8

    Tele (No)

    Tele (Yes)

    Telephone

    Detail Analysis Steps for 2*2 Factorial Design (5)

    LS Means plot for main effects

    • LS Means plot for categorical factors

    • Average acc bal of either direct mail or Telephone irrespective of other factor

    • Intended for ANOVA analysis

    • Prediction profiler gives similar results


    Detail analysis steps for 2 2 factorial design 6

    Interaction profiles

    18

    Mail(yes)

    16

    14

    Direct Mail

    Acc Bal

    Direct Mail

    12

    Mail(No)

    10

    8

    18

    Tele(Yes)

    16

    14

    Telephone

    Telephone

    Acc Bal

    12

    Tele(No)

    10

    8

    Mail(No)

    Tele(No)

    Mail(yes)

    Tele(Yes)

    Detail Analysis Steps for 2*2 Factorial Design (6)

    10

    Interaction plot is similar to LS Means plot


    Detail analysis steps for 2 2 factorial design 7

    Detail Analysis Steps for 2*2 Factorial Design (7)

    Cube plot

    11

    Tele (Yes)

    12

    16

    Telephone

    Tele (No)

    10

    14

    Direct mail

    Mail (No)

    Mail (yes)


    Summary of doe results in order of interest

    Summary of DOE Results in Order of Interest

    • Parameter estimates table – Estimates column

    • Pareto plot

    • Prediction profile plot

    • Main effect plot (LS Means plot) for categorical factors

      • Only available in effect leverage emphasis of Fit Model window

    • Interaction plot

      • Only if interaction specified in Fit Model window

    • Cube plot if number of factors is not too large (<3)

    Note: JMP contains additional, more advanced types of analysis for DOE results


    Checklist for factorial designs

    Checklist for Factorial Designs

    • Y– response is continuous

    • X – factors can be categorical or continuous that are run only at selected levels

      • X-factors are most often at 2 or 3 levels, but can have higher number of levels

      • X-factor levels are run in combination. Running all combinations results in full factorial experiment. Running a carefully selected subset results in fractional factorial experiment

      • Three X-factors each at 2 levels is a 2*2*2=23 design

      • Three X-factors two run at 2 levels and one at three levels is a 22*3 design


    Fractional factorial designs screening designs

    Fractional Factorial Designs – Screening Designs

    • Find important X factors affecting Y

      • Screening experiments

      • Highly Fractional Factorial experiments

    • Estimate Factor Effects & Interactions

      • Full Factorial experiments

      • RIV or higher resolution designs

    • Find Optimal Factor combinations

      • Response Surface


    Why we need screening designs

    Why we need Screening Designs

    • To eliminate any factors that could but don’t have an effect on the performance variable

      • Separate the vital few from the trivial many

      • Pareto principle: 80% of variation due to 20% of factors

    • To obtain focused information on the cause and effect relationship between factors and performance variable

      • Focused information may not be complete

    • To get maximum bang for minimum bucks!


    Fractional factorial designs at two levels screening designs in jmp

    Fractional Factorial Designs at two levels – Screening Designs in JMP

    • Factorial Designs can have any number of levels. For screening the suggested number is 2 levels

    • Number of experimental conditions is half, fourth, eight, etc. (powers of -2) of a full factorial design

    • Use fractional Factorial Designs instead of full Factorial Designs because

      • The number of factors k is so large as to require too many experimental conditions (2k) in terms of time and other resources

      • Higher order interactions are not of interest

      • Some of the Main Effects may not be of any importance, thus creating redundancy in the data


    Generating a screening design

    Generating a Screening Design

    • In the exploratory phase of experimentation, the number of experimental factors may be close to the number of runs

      • Nearly saturated or saturated designs

    • Number of levels for each factor is typically 2

    • Purpose of a Screening Design is to determine the most important Main Effects (X-factors)

    • Obtain design matrix from Screening Design

    1


    Processing time example

    Processing Time Example

    • Screening: Which of 6 factors are important?

    • Response: Processing time from application to decision in days

    • DOE: use 6 factors in 8 runs

      • Paper or digitized form of application

      • Staff level (10, 20)

      • Credit check (Normal, rapid)

      • Application is complete? (Yes or No)

      • Processing center (Old, new)

      • Loan amount (low, hi) with low anything below $1 million


    Selecting proper screening design 1

    1

    2

    Selecting Proper Screening Design (1)

    6 factors require a minimum of 8 runs!

    Select the Screening Design with 6 factors from a table in JMP


    Selecting proper screening design 2

    Selecting Proper Screening Design (2)

    3

    Enter factor names, types and ranges

    Data are in ProcessTime6Fac.jmp

    4. Generate matrix

    5. Obtain Y values from experiment

    4

    5


    Analyzing results using fit model 1

    OR

    1a

    1b

    2

    Add 6 factors as Model Effects

    Analyzing Results Using Fit Model (1)


    Analyzing results using fit model 2

    Full effectsDays

    1. Staff level -8.76

    2. Center-8.44

    3. App complete+4.68

    3

    4

    Analyzing Results Using Fit Model (2)

    How big is the factor effect?


    Analyzing results using fit model 3

    1

    3

    2

    Analyzing Results Using Fit Model (3)

    The steepest slopes

    • 1. Staff level

    • 2. Center

    • 3. App complete

    • These seem to be the most important factors. Run additional experiment to confirm results

    5

    Prediction profiler


    Pitfalls of screening experiments

    Pitfalls of Screening Experiments

    • Small number of observations may miss important factors

      • Limited resources may leave no choice but to run a few observations

    • Interactions may be impossible to evaluate

    • Factor setting may not be wide enough to show the area of interest or performance


    Next step after screening confirm

    Next Step After Screening – Confirm

    • Consider running a confirmatory experiment with

      • 2 factors (Staff, center) involving 4 runs, or

      • 3 factors (Staff, center, App complete) involving 8 runs

    • Do nothing and go with the results already discovered


    Full factorial designs

    Full Factorial Designs

    • Find important X factors affecting Y

      • Screening experiments

      • Highly Fractional Factorial experiments

    • Estimate Factor Effects & Interactions

      • Full factorial experiments

      • RIV or higher resolution designs

    • Find optimal factor combinations

      • Response Surface


    Full factorial design with 3 factors each at two levels 2 3

    Full Factorial Design with 3 factors each at two levels – 23

    • 1 overall mean

    • 3 Main effects A, B, C

    • 3 Two-Factor interactions AB, AC, BC

    • 1 Three-Factor interaction ABC

    • 2*2*2 = 8 different combinations possible →8 experimental runs

    Estimate the magnitude of each main effect and possible interaction effects!


    Factorial design matrix with three factors a b c

    Factorial Design Matrix with Three Factors A, B, C

    Each RUN (row) is an experimental condition

    Each Column shows the number of times a factor is run at high and low

    Column + and – are used to compute factor effects

    Run

    A

    B

    C

    Code

    1

    2

    3

    4

    5

    6

    7

    8

    -

    +

    -

    +

    -

    +

    -

    +

    -

    -

    +

    +

    -

    -

    +

    +

    -

    -

    -

    -

    +

    +

    +

    +

    (1)

    a

    b

    ab

    c

    ac

    bc

    abc

    Systematic collection of data based on DESIGN MATRIX


    Loan application time example

    Loan Application Time Example

    Only important factors are considered in this DOE

    • Factor A: Staffing level

      • 10 versus 20 staffers

    • Factor B: Center

      • Old versus new

    • Factor C: Application complete

      • Yes versus No

    • Response Y: Time to complete loan application

      • Hours

    Purpose: Determine which combination of these three factors results in the shortest time to complete the loan application


    Design and data of 3 important factors

    Design and Data of 3 Important Factors

    Factorial of 3 factors each at two levels

    Full factorial requires 8 = 2*2*2 runs

    Response variable = Time in days

    Factors

    Staff level10 or 20

    CenterOld or new

    Application completeYes or No


    Analysing a model in jmp

    2

    Analysing a Model in JMP

    1

    OR

    ProcessTime3Factor.jmp


    Parameter estimates pareto plot of factor effects

    Parameter Estimates & Pareto Plot of Factor Effects

    3

    4

    Eliminate staff level* center and staff level* app. complete interactions!


    Re analyse reduced factors for loan processing times

    Re-analyse Reduced Factors for Loan Processing Times

    Factor ABCY

    1

    Rerun observations or use old observations

    2

    Estimate Model terms with more precision by eliminating non-significantinteractions terms


    How much change due to each factor

    3

    All p-values at or below 0.05 level

    4

    Full effect-8.36+10.86-4.48

    Staff Center App Complete

    How much Change due to Each Factor?

    Important factors have steep prediction profile


    Cube plot in 2 3 experiment

    Cube plot in 23 Experiment

    30

    20.22

    11.85

    25

    20

    Old

    27.77

    19.41

    15

    Old

    Time LS Means

    10

    New

    b

    5

    0

    No

    Yes

    App Complete

    12.45

    4.08

    Yes

    a

    5

    13.84

    5.47

    No

    New

    App Complete

    10

    Staff Level

    20

    Best Performance = Shortest Time

    New Center with 20 Staff

    Model


    Pitfalls of factorial designs

    Pitfalls of Factorial Designs

    • Data have to much inherent variability to show the factor effects

      • Run replications of the same conditions and use averages

      • More costly to run replications

    • Factors can not be run in suggested random order

      • Run as is possible, but hedge interpretation appropriately

      • Do not use p-values

    • Non-linear (Curvilinear) relationships between Y and Xs prevalent

      • Use more than 2 levels for those factors

      • More runs require more resources


    Response surface designs

    Response Surface Designs

    • Find important X factors affecting Y

      • Screening experiments

      • Highly Fractional Factorial experiments

    • Estimate Factor Effects & Interactions

      • Full factorial experiments

      • RIV or higher resolution designs

    • Find optimal factor combinations

      • Response Surface


    Response surface designs1

    Response Surface Designs

    Why Response Surface methodology is needed

    • To Optimise the factor level settings

    • To find how sensitive an solution is to perturbations

      What Response Surface methodology can do

    • Designs that allow fitting quadratic curves to the data

    • Quadratic response curves can be used to find optimum value

    • Fairly advanced methodology beyond the scope of this course


    Exercise advertising mix

    Exercise – Advertising Mix

    Goal: Practice DOE Skills

    Instructions:

    • For an advertising campaign of a new bank product a bank wants to estimate the optimum mix of advertising media. The bank considers

      • Print media: Full page and half-page ads

      • Television commercials: 30 and 60 second spots

      • Radio commercials: 1 minute and 2 minute commercials

    • The bank operates in many local markets so that it is in a position to test its media mix for effectiveness

    • Design an experiment to evaluate the mix, specifically answer:

      • How many test markets are required

      • Design the advertising mix for each market

      • Look at design alternatives and discuss their pros and cons

    Time:Teams:30 minutes

    Report out:15 minutes


    Solution to advertising mix 1

    Solution to Advertising Mix (1)

    • From DOE select Screening Designs

    • Treat all four factors as Categorical

      • Print media: full and half page adds. Are the only two sizes mentions. Newspapers sell adds at discrete increments

      • TV and Radio commercials also come in fixed lengths. It is not possible to have a 37 sec TV commercial, etc. Even though one could think of a 37 sec commercial, the length are standard and in between are not used. Therefore even the continuous time is discretized to a categorical factor

      • Billboards & Buses are either included or not

    • Enter 4 categorical factors by clicking corresponding Add button

    • Enter values for each factor. (This step is not necessary to obtain output but it is highly recommended if you want to understand the output)

    • Click Continue


    Solution to advertising mix 2

    Solution to Advertising Mix (2)

    • Fractional Factorial Designs highlights simplest design with 8 runs (no blocks)

    • Use only 8 test markets, because fractional Factorial Designs are available only in powers of 2. A ninth test market would add very little to the analysis and more likely would complicate analysis

    • Run Order: for review choose left to right to right to left. Fir performing the actual experiment choose Randomize to scramble the run order

    • Click on Make Table and a data table will result


    Solution to advertising mix 3

    Solution to Advertising Mix (3)

    • The Data table below shows four factor level combinations for each of the 8 runs

      • Run 1: Full page print, 30 sec TV, 1 min Radio, no B&B

      • Run 2: Full page print, 30 sec TV, 2 min Radio, yes B&B

      • etc

    • The next step is to perform the advertising test in all 8 test markets. Enter the results in the so far empty Y-column

    • Click on red triangle next to Model in Fractional Factorial window and the data will be analysed in ANOVA like fashion


    Summary8

    Summary

    • DOE allows efficient and systematic evaluation of many factors simultaneously

    • Factors are estimated independently of each other due to the way data are collected

    • DOE can be used for separating important factors from unimportant ones→Screening experiments

    • DOE can be used to estimate the impact of important factors? Full Factorial Designs

    • DOE could also be used for Optimisation of factor combinationsinvolving continuous factors? Response Surface Designs


    Analyse phase7

    Analyse Phase

    Module 1

    Recap of the Measure Phase

    Module 2

    Overview of the Analyse Phase

    Module 3

    Graphical Data Analysis

    Module 4

    Simple – Identify, Rank and Validate Key X’s

    - 5 Why

    - Cause and effect diagram

    - Multi-voting

    Module 5

    Validate the Vital Few

    - One sample methods

    - Two sample methods

    - Chi-Square

    Module 6

    More Advanced – Identify, Rank and Validate Key X’s

    - ANOVA

    - Simple regression

    Advanced – Identify, Rank and Validate Key X’s

    - Introduction to design of experiments

    Module 7

    Module 8

    Tollgate


    Champion s tollgate guide

    Champion’s Tollgate Guide

    Analyse

    • Ensure project linkage to business strategy

    • Determine root cause(s) & significant sources of variation that are impeding the desired performance

    Probing question(s)

    What to look for?

    • Show me your cause and effect analysis

    • What are the elements/factors driving or controlling the requirements or desired outcome?

    • How was it analysed?

    • The basis for the root cause, plausibility of the logic. Whether or not a symptom is being pursued, instead of root causes

    • The relationship between the cause and influence on the problem

    • Good use of the cause & effect or other relevant analytical tools

    Critical checkpoints

    Phase objective

    • Strategy for determining Causal System

    • Verify Root Cause

    • Isolate key X’s that will eliminate defects

    • Perform graphical Data Analysis

    • Conduct statistical Analysis

      • Hypothesis Testing

      • Regression Analysis

    • Identifying the critical factors driving the requirement(s) (Y’s)

    • Identify improvement impact

    • Find root cause(s) of variation

    • How much of the problem, as described in the Measure phase are you going after?

    • Which are the root causes? How do you know?

    • How did you determine their impact?

    • What was expected?

    • Good segmentation when problem is big or complex

    • The kind of correlation or test developed to be able turn on or off the symptoms

    • Demonstration or validation of project impacts or benefits

    • Whether or not additional work will be required and a plan for going forward

    Recommended tools

    • Is there an analysis of the relative importance of the X’s?

    • Are there newly identified secondary metrics?

    • Does the process owner buy into these root causes?

    • Assessment of the strength of the analysis if tools such as Design of Experiment (DOE) or Regression

    • Evidence that the proposed changes are not having negative side effects. Check back to the secondary metric performance and check for stability

    • Adequate segmentation was done. Good segmentation can provide insights into potential improvements

    • Right level of engagement of the process owner is being formed. This will facilitate an easier transition and minimise surprises. Credibility with the process owner of what is being presented is key

    • C-E Diagram1

    • Pareto1

    • Chi-Square

    • DOE

    • Scatter Diagram1

    • Regression

    • Non-value-added

    • Hypothesis Tests

    Do’s

    Don'ts

    • Encourage the team to continue to persevere

    • Provide insightful comments

    • Be supportive and help remove roadblocks

    • Use data... Use data...

    • Be closed minded

    • Give unrealistic implementation stretch goals

    • Use insights to punish

    • Are there any issues/barriers that prevent you from completing the analyse phase?

    • Do you still have adequate resources to complete the project?

    • Are there any immediate improvements we can make?

    • Opportunity to help the team overcome obstacles

    • Provide additional resources if needed

    • The kind of sequencing that will help expedite the rate of implementation without compromising a well disciplined approach

    (1) Strongly recommended for Tollgate


    Hsbc six sigma black belt training analyse

    Reqt’s

    1.0

    2.0

    0

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    +

    D

    M

    A

    I

    C

    Prioritised list of improvements (X’s)

    • Hhhh

    • Khknknk

    • Llflflfl

    • mnmnkf

    Factors impacting requirements (Y’s)

    • Cause & effect

    • Pareto

    • Scatter diagram

    • ABC...

    Comments

    Opportunity assessment

    • Priority

    • Improvement goal for Y

    Validation/verification of (X’s)

    • Hypothesis

    • Root cause

    Lessons learned

    • Sscc

    • Sssss

    • Ssss

      What we need

    • mvvmvm

    • Sdfgh

    • Sklfg


    Hsbc six sigma black belt training analyse

    Major elements (sub-process) with

    requirements

    Key takeaways

    D

    M

    A

    I

    C


    Analyse identity root causes and priority improvement opportunities

    Receive request

    Proposal prep

    Approval

    D

    M

    A

    I

    C

    Analyse – Identity root causes and priority improvement opportunities

    List of priority improvements (X’s)

    Factors impacting outputs (Y’s)

    1.Time consuming applications

    2.Decision cycle time with re-work

    3.Decline process – Large

    4.Resources consumed renewals (ABC)

    Comments

    1.Reducing transmission time will provide immediate cycle time improvement

    2.For Large credits, re-work loop is consuming 10 days cycle time 1/3 of the time

    3.Large decline rate: 42%. Resources consumed to decline Large deals: $4.0m per year

    4.Resources consumed to process Large renewals: $3.2m per year

    5.Process hand-offs eliminated

    6.Reducing transmission time will provide immediate cycle time improvement

    Small

    Medium

    Large

    1. Resolve submittal of applications through mail

    2. Reduce DCO additional information requests

    3. Move kill point for unapprovable applications earlier in process, before reaching DCO’s. Also institute better strike zoning

    Current kill point deals declined

    4. Reduce resources consumed in renewals process

    5. Reconfigure personnel and process into teams including underwriting, support and documentation

    6. Resolve transmission of documents through mail

    • Opportunity assessment

    • Reasons for selecting Large as priority for initial improvements:

      • 63% of the application dollars

      • Highest resources consumed

      • Area with most noise/complaints

    • Business processes prioritised

    • Potential improvement goal

      • Expected gains 2-3 days decision, 6-8 days booking cycle times

    • Validation/verification of (X’s)

      Root causes

      Used Case and Effect analysis to confirm

      major causes (see Appendix I)

    • Incomplete applications driving higher cycle times

    • Applications for which additional information is requested by DCO/Underwriter have higher median cycle times than those not

    • Mail time significant cycle time factor

      All tests of medians conducted at 95% confidence

    • Lessons learned

    • Costs revised up to $33.2 million.See Appendix II

    • Approx.. 20% of overall cycle time is consumed by mail time of applications and documents

    • Team needs

    • Support for personnel resources and other investments required particularly in the IT area

    • Approval to move to Improve phase

    Receive requests, analysis and proposal preparation

    Credit approval

    Documentation request and prep

    Doc transit, closing and booking

    % of total Large cycle time consumed by processes above:

    27%

    31%

    12%

    30%

    Improvement targeted to circles areas!


    Crip ii

    2,000

    1,500

    1,000

    500

    0

    Count

    Question 4a, time for ann. rev

    Question $b, CAT A request

    Question 4c, CAT B request

    Question 4d, time for admin CARM

    100

    80

    60

    40

    20

    0

    Cum percent

    NVA

    D

    M

    A

    I

    C

    Crip II

    Areas of focus:

    • Seek approval to establish unallocated limits during annual review to cover interim needs (attached as Appendix A)

    • Reduce the # of Interim and Admin CARMs RMs/TLs must see. Interim requests and Admin CARMs account for over 50% of RM time. Team recommends addressing this by expanding the role CSU plays in CARM prep for interim and admin CARMs and defining what CARMs can move from Analysts to Credit

    • Minimise RM time spent on NVA and BVA activities

    • Reduce the absolute # of CARMs and RMs touch

    Which CARM steps consume RM time?

    Team hosted brainstorming session with other CIB staff

    • Team focused on NVA steps (rework): What are the likely causes of rework?

    Specific improvements have been identified

    What we need:

    • Continued Senior Management support as the team seeks GHQ approval for the Unallocated Limit Proposal

    • Continued co-operation from Credit Policy to consider modifications to CARM

    • Approval to commence implementation of improvements

    • Root causes of NVA were identified and validated by the Team. Next step: identifying solutions


    Appendix 1 glossary of terms

    Appendix 1Glossary of terms


    Glossary of terms a c

    BLOCKING VARIABLES

    A relatively homogenous set of conditions within which different

    conditions of the primary variables are compared. Used to ensure that

    background variables do not contaminate the evaluation of primary

    variables

    ABSCISSA

    The horizontal axis of a graph

    ACCEPTANCE REGION

    The region of values for which the Null Hypothesis is accepted

    ALPHA RISK

    The probability of accepting the alternate hypothesis when, in reality, the Null Hypothesis is true

    BRAINSTORMING

    A team-oriented meeting used in problem solving to develop a list of

    possible causes that may be linked to an observed effect

    ALTERNATIVE HYPOTHESIS

    A tentative explanation which indicates that an event does not follow a

    chance distribution; a contrast to the Null Hypothesis

    CAPABILITY INDICES

    A mathematical calculation used to compare the process variation to a

    specification. Examples are Cp, Cpk, Pp, PpK, Zst, and Zlt. General

    Electric uses Zst and Zlt as the common communication language on

    process capability

    ANALYSIS OF VARIANCE

    A statistical method for evaluating the effect that factors have on

    process mean and for evaluating the differences between the means of two or more normal distributions

    CAUSALITY

    The principle that every change implies the operation of a cause.

    ASSIGNABLE CAUSE

    A process input variable that can be identified and that contributes in an observable manner to non-random shifts in process mean and/or standard deviation

    CAUSATIVE

    Effective as a cause

    CAUSE

    That which produces an effect or brings about a change

    ASSIGNABLE

    VARIATIONS

    Variations in data which can be attributed to specific causes

    CAUSE AND EFFECT

    (C&E) DIAGRAM

    One of the seven basic tools for problem solving and is sometimes

    referred to as a "fishbone" diagram because of its structure. Spine

    represents the "effect" and the major legs of the structure are the "cause

    categories.” The substructure represents the list of potential causes which can induce the "effect." The 6 M's (men & women, machine, material, methods, measurements, and Mother Nature) are sometimes used as cause categories

    ATTRIBUTE DATA

    Quality data that typically reflects the number of conforming or non-conforming units or the number of non-conformities per unit on a go/no go or accept/reject basis

    AVERAGE

    Sum of all measurements divided by the total number of measurements. Statistic which is used to estimate the population mean. Same as MEAN

    BACKGROUND

    VARIABLES

    Variables which are of no experimental interest and are not held

    constant. Their effects are often assumed insignificant or negligible, or they are randomised to ensure that contamination of the primary response does not occur. Also referred to as environmental variables and uncontrolled variables

    C CHARTS

    Charts which display the number of defects per sample. Used where

    sample size is constant

    CENTRAL TENDENCY

    Numerical average, e.g., mean, median, and mode; centre line on a

    statistical process control chart

    BENCHMARKING

    A process for identification of external best-in-class practices and standards for comparison against internal practices

    CENTER LINE

    The line on a statistical process control chart which represents the

    characteristic's tendency

    BETA RISK

    The probability of accepting the Null Hypothesis when, in reality, the

    alternate hypothesis is true

    CHAMPION

    An executive level business leader who facilitates the leadership,

    implementation, and deployment of Six Sigma philosophies

    BINOMIAL

    DISTRIBUTION

    A statistical distribution associated with data that is one of two possible

    states such as Go-No Go or Pass-Fail

    CHANGE ACCELERATION

    PROGRAM (CAP)

    A process which helps accelerate stakeholder buy-in and implementation

    of new philosophies and processes within a business

    BLACK BELT

    A process improvement project team leader who is trained and certified

    in Six Sigma methodology and tools and who is responsible for

    successful project execution

    CHARACTERISTIC

    A definable or measurable feature of a process, product, or service

    Glossary of terms A-C


    Glossary of terms c f

    CRITICAL TO QUALITY

    (CTQ) CHARACTERISTIC

    A drawing characteristic determined to be important for variability

    reduction based on a requirement from production, engineering, customer

    application, or regulatory agency. Can also apply to transactional or

    service delivery processes

    CLASSIFICATION

    Differentiation of variables

    COMMON CAUSE

    See RANDOM CAUSE

    CONFIDENCE LEVEL

    The probability that a randomly distributed variable "x" lies within a

    defined interval of a normal curve

    CUTOFF POINT

    The point which partitions the acceptance region from the reject region

    The two values that define the confidence interval

    CONFIDENCE LIMITS

    DATA

    Factual information used as a basis for reasoning, discussion, or

    calculation; often refers to quantitative information

    CONFOUNDING

    Allowing two or more variables to vary together so that it is impossible to

    separate their unique effects

    DATA TRANSFORMATION

    A mathematical technique used to create a near normally distributed data

    set out of a non-normal (skewed) data set

    CONSUMER’S RISK

    Probability of accepting a lot when, in fact, the lot should have been

    rejected (see BETA RISK)

    DEFECT

    Any product characteristic that deviates outside of specification limits

    DEFECT PER MILLION

    OPPORTUNITIES (DPMO)

    Quality metric used in the Six Sigma process and is calculated by the

    number of defects observed divided by the number of opportunities for

    defects normalised to 1 million units

    CONTINUOUS DATA

    Data obtained from a measurement system which has an infinite number

    of possible outcomes

    A random variable which can assume any value continuously within

    some specified interval

    CONTINUOUS RANDOM

    VARIABLE

    DESIGN FOR CUSTOMER

    IMPORT (DFCI)

    Approach to customers characterised by customer centricity and

    measuring from the customer point-of-view

    CONTROL CHART

    A graphical rendition of a characteristic's performance across time in

    relation to its natural limits and central tendency

    FAILURE MODE &

    EFFECTS ANALYSIS

    (FMEA)

    Analytical technique focused at problem prevention through

    identification of potential problems. The FMEA is a proactive tool that

    is used pragmatically to identify potential failures and their effects, to

    numerically rate the combined risk associated with severity, probability

    of occurrence and delectability, and to document appropriate plans for

    prevention. FMEA’s can be applied to system, application, and product

    design and to manufacturing and non-manufacturing processes (i.e.,

    services and transactional processes)

    CONTROL LIMITS

    Apply to both range or standard deviation and subgroup average (X)

    portions of process control charts and are used to determine the state of

    statistical control. Control limits are derived statistically and are not

    related to engineering specification limits in any way

    CONTROL PLAN

    A formal quality document that describes all of the elements required to

    control variations in a particular process or could apply to a complete

    product or family of products

    FIRST TIME YIELD

    Yield that occurs in any process step prior to any rework that may be

    required to overcome process shortcomings

    Specification requirements for the product being manufactured

    CONTROL

    SPECIFICATIONS

    FIXED EFFECTS MODEL

    An experimental model where treatments are specifically selected by the

    researcher. Conclusions only apply to the factor levels considered in the

    analysis. Inferences are restricted to the experimental levels

    CORRELATION

    The relationship between two sets of data such that when one changes,

    the other is likely to make a corresponding change. Also, a statistical tool

    for determining the relationship between two sets of data

    FLUCTUATIONS

    Variances in data which are caused by a large number of minute

    variations or differences

    COST OF POOR QUALITY

    (COPQ)

    Cost associated with providing poor quality products or services. Can be

    divided into four cost categories: Appraisal, Scrap, Rework, and Field

    Complaint (warranty costs)

    FREQUENCY

    DISTRIBUTION

    The pattern or shape formed by the group of measurements in a

    distribution based on frequency of occurrence

    Glossary of terms C-F


    Glossary of terms g m

    A measure of the variation observed when a single operator uses a gage

    to measure a group of randomly ordered (but identifiable) parts on a

    repetitive basis

    A measure of average variation observed between operations when

    multiple operators use the same gage to measure a group of randomly

    ordered (but identifiable) parts on a repetitive basis

    HOMOGENEITY OF

    VARIANCE

    GAGE REPEATABILITY

    GAGE REPRODUCIBILITY

    A measure of variation observed when a gage is used to measure the

    same master over an extended period of time

    A measure of gage accuracy variation when evaluated over the expected

    operating range

    The average difference observed between a gage under evaluation and a

    master gage when measuring the same parts over multiple readings

    Six Sigma role similar in function to Black Belt, but length of training

    and project scope are reduced

    Vertical display of a population distribution in terms of frequencies; a

    formal method of plotting a frequency distribution

    The average difference observed between a gage under evaluation and a

    master gage when measuring the same parts over multiple readings

    The variances of the data groups being contrasted are equal (as defined

    by a statistical test of significant difference)

    LINE CHARTS

    MISTAKE-PROOFING

    GAGE ACCURACY

    GAGE LINEARITY

    MEAN TIME BETWEEN

    FAILURES (MTBF)

    LOWER CONTROL LIMIT

    MEASUREMENT SYSTEMS

    ANALYSIS (MSA)

    INTERVAL

    GAGE STABILITY

    GAGE ACCURACY

    INDEPENDENT VARIABLE

    INTERACTION

    MEDIAN

    INSTABILITY

    MEAN

    MIXED EFFECTS MODEL

    HISTOGRAM

    MINITAB

    GREEN BELT

    MULTI-VARI

    Numeric categories with equal units of measure but no absolute zero

    point, i.e., quality scale or index

    A horizontal dotted line plotted on a control chart which represents the

    lowest process deviation that should occur if the process is in control (free from assignable cause variation)

    See AVERAGE

    Average time to failure for a statistically significant population of a

    product operating in its normal environment

    A controlled variable; a variable whose value is independent of the value

    of another variable

    Unnaturally large fluctuations in a process input or output characteristic

    Contains elements of both the fixed and random effects models

    Statistical software package that operates on Microsoft Windows with a

    spreadsheet format and has powerful statistical analysis ability

    Charts used to track the performance without relationship to process

    capability or limits

    Means of evaluating a continuous or discrete measurement system to

    quantify the amount of variation contributed by the measurement system.

    Refer to Automotive Standard (AIAG STD) for details

    Method used in the measure/analyse phase of Six Sigma to display in

    graphical terms variation within parts, machines, or processes between

    machines or process parts, and over time

    Mistake proofing is a proactive technique used to positively prevent errors

    from occurring

    The mid-value in a group of measurements when ordered from low to

    high

    The tendency of two or more variables to produce an effect in

    combination which neither variable would produce if acting alone

    KEY PROCESS INPUT

    VARIABLES (KPIV’s)

    The vital few input variables, called "x’s" (normally 2-6), that drive 80%

    of the observed variations in the process output characteristic ("y")

    MASTER BLACK BELT

    A person who is an "expert" on Six Sigma techniques and on project

    implementation. Master Black Belts play a major role in training,

    coaching and in removing barriers to project execution in addition to

    overall promotion of the Six Sigma philosophy

    HYPOTHESIS

    When used as a statistical term, it is a theory proposed or postulated for

    comparing means and standard deviations of two or more data sets. A

    "null" hypothesis states that the data sets are from the same statistical

    population, while the "alternate" hypothesis states that the data sets are not from the same statistical population

    Glossary of terms G-M


    Glossary of terms n p

    NULL HYPOTHESIS

    NORMAL DISTRIBUTION

    PROCESS AVERAGE

    PROBABILITY

    POISSON DISTRIBUTION

    PROBLEM

    PROBABILITY OF AN EVENT

    NONCONFORMING UNIT

    NORMALIZED ROLLED

    THROUGHPUT YIELD (RTYN)

    POWER OF AN

    EXPERIMENT

    PARETO DIAGRAM

    PRECISION TO

    TOLERANCE RATIO (P/T)

    P CHARTS

    POPULATION

    ONE-SIDED

    ALTERNATIVE

    PROBLEM-SOLVING

    OUT OF CONTROL

    ORDINATE

    POPULATION

    PROCESS CONTROL CHART

    PERTURBATION

    PROCESS CONTROL

    ORDINAL

    PROCESS

    PREVENTION

    PARAMETER

    The probability of rejecting the Null Hypothesis when it is false and

    accepting the alternate hypothesis when it is true

    A chart which places common occurrences in rank order

    The entire set of items from which a sample is drawn

    The estimate of the average process yield used to determine RTY. It is

    determined by taking the nth root of the RTY (where "n" is the # of

    process steps) included in the RTY calculation

    A ratio used to express the portion of engineering specification consumed

    by the 99% confidence interval of measurement system repeatability and

    reproducibility error. (5.15 standard deviations of R&R error)

    An assertion to be proven by statistical analysis where two or more data

    sets are stated to be from the same population

    The practice of eliminating unwanted variation before the fact, e.g.,

    predicting a future condition from a control chart and then applying

    corrective action before the predicted event transpires

    Ordered categories (ranking) with no information about distance between

    each category, i.e., rank ordering of several measurements of an output

    parameter

    A non-random disturbance

    The vertical axis of a graph

    A continuous, symmetrical density function characterised by a bell-shaped curve, e.g., distribution of sampling averages

    The value of a parameter which has an upper bound or a lower bound, but

    not both

    A statistical distribution associated with attribute data (the number of non-conformities found in a unit) and can be used to predict first pass yield

    The process of solving problems; the isolation and control of those

    conditions which generate or facilitate the creation of undesirable

    symptoms

    Condition which applies to statistical process control chart where plot

    points fall outside of the control limits or fail an established run or trend

    criteria, all of which indicate that an assignable cause is present in the

    process

    Charts used to plot percent defectives in a sample where sample size is

    variable

    The central tendency of a given process characteristic across a given

    amount of time or at a specific point in time

    A unit which does not conform to one or more specifications, standards,

    and/or requirements

    See STATISTICAL PROCESS CONTROL

    The number of successful events divided by the total number of trials

    A group of similar items from which a sample is drawn. Often referred to

    as the universe

    The chance of an event happening or condition occurring by pure chance

    and is stated in numerical form

    A deviation from a specified standard

    A constant defining a particular property of the density function of a

    variable

    Any of a number of various types of graphs upon which data are plotted

    against specific control limits

    A particular method of doing something, generally involving a number of

    steps or operations

    NONCONFORMITY

    A condition within a unit which does not conform to some specific

    specification, standard, and/or requirement; often referred to as a defect;

    any given non-conforming unit can have the potential for more than one

    nonconformity

    PRIMARY CONTROL

    VARIABLES

    The major independent variables used in the experiment

    Glossary of terms N-P


    Glossary of terms p r

    PROCESS MAP

    A detailed step-by-step pictorial sequence of a process showing process

    inputs, potential or actual controllable and uncontrollable sources of

    variation, process outputs, cycle time, rework operations, and inspection

    points

    PROCESS SPREAD

    The range of values which a given process characteristic displays; this

    particular term most often applies to the range but may also encompass the variance. The spread may be based on a set of data collected at a specific point in time or may reflect the variability across a given period of time

    RANDOM CAUSE

    A source of variation which is random, usually associated with the "trivial

    many" process input variables, and which will not produce a highly

    predictable change in the process output response (dependent variable),

    e.g., a correlation does not exist; any source of variation results in a small

    amount of variation in the response; cannot be economically eliminated

    from a process; an inherent natural source of variation

    RANGE

    RESEARCH

    RANDOM VARIABLE

    PRODUCER’S RISK

    ROOT SUM SQUARED (RSS)

    PROJECT

    RANDOM SAMPLE

    ROLLED THROUGHPUT

    YIELD (RTY)

    RATIONAL SUBGROUP

    RANDOMNESS

    R CHART

    RESIDUAL ERROR

    RANDOM VARIATIONS

    RESPONSE SURFACE

    METHODOLOGY (RSM)

    REPRESENTATIVE

    SAMPLE

    RANK

    ROBUST

    QUALITY FUNCTION

    DEPLOYMENT (QFD)

    REGRESSION

    REJECTION REGION

    REPLICATION

    Plot of the difference between the highest and lowest in a sample.

    Normally associated with the range control portion of an X, R chart

    The condition or state in which a response parameter exhibits a high

    degree of resistance to external causes of a non-random nature; i.e.,

    impervious to perturbing influence

    Probability of rejecting a lot when, in fact, the lot should have been

    accepted (see ALPHA RISK)

    A problem, usually calling for planned action

    Selecting a sample such that each item in the population has an equal

    chance of being selected; lack of predictability; without pattern

    Critical and exhaustive investigation or experimentation having for its aim

    the revision of accepted conclusions in the light of newly discovered facts

    See EXPERIMENTAL ERROR

    A statistical technique for determining the best mathematical expression

    that describes the functional relationship between one response and one or more independent variables

    Repeat observations made under identical test conditions

    The difference between the highest and lowest values in a "subgroup"

    sample

    A subgroup is usually made up of consecutive pieces chosen from the

    process stream that the variation represented within each subgroup is as

    small as feasible. Any changes, shifts and drifts in the process will appear

    as differences between the subgroups, selected over time

    A variable which can assume any value from a distribution which

    represents a set of possible values

    A graphical (pictorial) analysis technique used in conjunction with DOE

    for determining optimum process parameter settings

    A sample which accurately reflects a specific condition or set of

    conditions within the universe

    Square root of the sum of the squares. Means of combining standard

    deviations from independent causes

    The product (series multiplication) of all of the individual first pass yields

    of each step of the total process

    Values assigned to items in a sample to determine their relative occurrence in a population

    The region of values for which the alternate hypothesis is accepted

    Variations in data which result from causes which cannot be pinpointed or

    controlled

    A condition in which any individual event in a set of events has the same

    mathematical probability of occurrence as all other events within the

    specified set, i.e., individual events are not predictable even though they

    may collectively belong to a definable distribution

    QFD is a disciplined matrix methodology used for documenting and

    transforming customer wants and needs – "the voice of the customer" -

    into operational "requirement" terms. It is an effective tool for determining

    critical-to-quality characteristics for transactional processes, services and

    products

    Glossary of terms P-R


    Glossary of terms s u

    SAMPLE

    A portion of a population of data chosen to estimate some characteristic

    about the whole population. One or more observations drawn from a larger collection of observations or universe (population)

    SUBGROUP

    A logical grouping of objects or events which displays only random event-to-event variations, e.g., the objects or events are grouped to create

    homogenous groups free of assignable or special causes. By virtue of

    minimising within subgroup variability, any change in the central tendency

    or variance of the universe will be reflected in the "subgroup-to-subgroup"

    variability

    A predetermined sample of consecutive parts or other data bearing objects removed from the process for the purpose of data collection

    SCATTER DIAGRAMS (PLOTS)

    Charts which allow the study of correlation, e.g., the relationship between

    two variables or data sets

    TYPE I ERROR

    STATISTICAL CONTROL

    UPPER CONTROL LIMIT

    SYSTEM

    UNNATURAL PATTERN

    THEORY

    STATISTICAL PROCESS

    CONTROL (SPC)

    TEST OF SIGNIFICANCE

    SIPOC

    SHORT RUN STATISTICAL

    PROCESS CONTROL

    SYMPTOM

    SYSTEMATIC VARIABLES

    SIX SIGMA

    TWO-SIDED

    ALTERNATIVE

    SYSTEM

    STABLE PROCESS

    SIX M'S

    SPECIAL CAUSE

    TYPE II ERROR

    SKEWED DISTRIBUTION

    STANDARD DEVIATION

    That which serves as evidence of something not fully understood in factual terms

    The values of a parameter which designate both an upper and lower bound

    That which is connected according to a scheme.

    A plausible or scientifically acceptable general principle offered to explain

    phenomena

    A statistical control charting technique which applies to any process

    situation where there is insufficient frequency of subgroup data to use

    traditional control charts (typically associated with low-volume

    manufacturing or where set-ups occur frequently). Multiple part numbers

    and multiple process streams can be plotted on a single chart

    A high-level process map. Stands for Supplier-Inputs-Process-Outputs-Customer

    A term coined by Motorola to express process capability in parts per

    million. A Six Sigma process generates a maximum defect probability of

    3.4 parts per million (PPM) when the amount of process shifts and drifts

    are controlled over the long-term to less than +1.5 standard deviations

    A non-symmetrical distribution having a tail in either a positive or

    negative direction

    See ASSIGNABLE CAUSE

    See ALPHA RISK

    The application of standardised statistical methods and procedures to a

    process for control purposes

    A horizontal line on a control chart (usually dotted) which represents the

    upper limits of capability for a process operating with only random

    variation

    That which is connected according to a scheme

    A process which is free of assignable causes, e.g., in statistical control

    Any pattern in which a significant number of the measurements do not

    group themselves around a central tendency. When the pattern is

    unnatural, it means that non-random disturbances are present and are

    affecting the process

    A statistical procedure used to determine whether or not a process

    observation (data set) differs from a postulated value by an amount

    greater than that due to random variation alone

    A statistical index of variability which describes the process spread or

    width of a distribution

    A pattern which displays predictable tendencies

    A quantitative condition which describes a process that is free of

    assignable/special causes of variation (both mean and standard deviation). Such a condition is most often evidenced on a control chart, i.e., a control chart which displays an absence of non-random variation

    The major categories that contribute to effects on the fishbone diagram

    (men & women, machine, material, methods, measurement, and Mother

    Nature

    See BETA RISK

    Glossary of terms S-U


    Glossary of terms v z

    VARIABLE

    VARIABLES DATA

    VARIATION

    VARIATION RESEARCH

    VOICE OF THE CUSTOMER

    X& R CHARTS

    A characteristic that may take on different values

    Data collected from a process input or output where the measurement scale has a significant level of subdivisions or resolution., e.g., ohms, voltage, diameter, etc

    Any quantifiable difference between individual measurements; such differences can be classified as being due to common causes (random or special causes (assignable)

    Procedures, techniques, and methods used to isolate one type of variation from another (for example, separating product variation from test variation)

    Data gathered from the customers that provides information about their needs and requirements

    A control chart which is a representation of process capability over time; displays the variability in the process average and range

    Glossary of terms V-Z


    Appendix 2 six sigma tool selector

    Appendix 2Six Sigma tool selector


    Hsbc six sigma tool selector1

    HSBC Six Sigma tool selector


  • Login