training objectives l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Training Objectives PowerPoint Presentation
Download Presentation
Training Objectives

Loading in 2 Seconds...

play fullscreen
1 / 122

Training Objectives - PowerPoint PPT Presentation


  • 422 Views
  • Uploaded on

Training Objectives Prepare you to plan for data collection and analysis by using the ASA Data Collection Plan Discuss various ways to display and analyze data Describe how to construct various graphs as they apply to the ASA Template Review the basics of creating charts using Excel

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Training Objectives' - Faraday


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
training objectives
Training Objectives
  • Prepare you to plan for data collection and analysis by using the ASA Data Collection Plan
  • Discuss various ways to display and analyze data
  • Describe how to construct various graphs as they apply to the ASA Template
  • Review the basics of creating charts using Excel
plan before you act
PLAN before you act!
  • Data collection can be time consuming
  • Need to figure out the Who, What, Where, When, Why for each performance measure
  • Distribute the data collection responsibility among your ASA Team Members
  • Garbage in = Garbage out
  • Plan now to prevent crisis down the road
slide4

ASA Data Collection Plan

  • Takes ASA Template information and spells out the specifics of gathering data
    • Performance objective
    • Performance measure
    • Methodology
    • Point of Contact (POC)
    • Source of Data
    • Frequency
    • Target
  • Recommend EACH ASA Team complete ONE Data Collection Plan
    • Road map for the team’s data collection efforts
    • Distributes responsibilities among team members
    • Allows the team to communicate their methods to others
  • ASA Consultant, OQM, your Division/Office Director may request to review your Data Collection Plan
slide5

Completing the ASA Data Collection Plan

  • Step 1: Review Common Performance Objectives and Measures
  • Step 2: Add Unique Performance Objectives and Measures
  • Step 3: Review/Determine Methodology
  • Step 4: List Points of Contact (POC)
  • Step 5: List Source of Data
  • Step 6: List Data Collection Frequency
  • Step 7: List Target
slide6

Completing the ASA Data Collection Plan (cont.)

  • Common performance objectives and performance measures are already listed
  • Need to add Discrete Services as appropriate
  • Need to add unique performance objectives and measures as appropriate
  • Boxes with text can be edited if desired
  • Shaded areas typically do not need to be filled in
slide7

Step 1: Review Common Performance Objectives and Measures

  • Review common measures in each perspective
  • If measurement is occurring at the Discrete Service level
    • Enter your Discrete Services on the form
    • Plan provides for 3 Discrete Services for common measures
    • Add/delete rows for Discrete Services as necessary
  • If measurement will occur at the Service Group level
    • Change shaded boxes to white background on row that lists the common measure
    • Enter information in that row
    • Delete the rows for the Discrete Services
step 1 review common performance objectives and measures cont
Step 1: Review Common Performance Objectives and Measures (cont.)

Shaded areas usually require no entry

Common performance measures from ASA Template are listed for each of the four perspectives.

step 1 review common performance objectives and measures cont9
Step 1: Review Common Performance Objectives and Measures (cont.)

Type in Discrete Services for common measures as appropriate.

step 1 review common performance objectives and measures cont10
Step 1: Review Common Performance Objectives and Measures (cont.)
  • To add rows
    • Click left mouse button on DS3
  • Choose “Insert”, “Row”
  • Select Columns A and B in new row
  • Choose “Format”, “Cells”, “Alignment”, “Merge Cells”
  • Click “OK”
  • Re-label and edit rows as appropriate
step 1 review common performance objectives and measures cont11
Step 1: Review Common Performance Objectives and Measures (cont.)
  • To delete rows
    • Click left mouse button on DS3
  • Choose “Edit”, “Delete”
  • Click “Entire Row”
  • Click “OK”
step 2 add unique performance objectives and measures
Step 2: Add Unique Performance Objectives and Measures

Type in Unique Performance Objectives and Measures on Template for each perspective. Add Discrete Services as needed.

step 3 review determine methodology
Step 3: Review/Determine Methodology
  • Methodology is a summary of how you plan to gather and analyze the data
    • For common measures, review the suggested methodology and edit as appropriate
    • For unique measures, decide on appropriate methodology and type on template
step 3 review determine methodology cont
Step 3: Review/Determine Methodology (cont.)

Edit or develop methodology for data collection and analysis.

step 3 review determine methodology cont15
Step 3: Review/Determine Methodology (cont.)
  • Edit methodology as appropriate
  • To resize rows
    • Position mouse at bottom of row number at left of screen
    • A crosshair appears
step 3 review determine methodology cont16
Step 3: Review/Determine Methodology (cont.)

Height: 236.25

  • Click left mouse button to display row height
  • Move mouse up or down to increase or decrease row height
step 3 review determine methodology cont17
Step 3: Review/Determine Methodology (cont.)
  • To format cells for word-wrap
    • Select cell(s) with mouse
    • Choose “Format”, “Cells” , “Alignment”
    • Click “Wrap text”, “Merge cells”
    • Click “OK”
step 4 list points of contact poc
Step 4: List Points of Contact (POC)
  • Decide who will have primary responsibility for data collection and analysis for each performance measure
    • Serve as focal point regarding the data collection/analysis if several people are involved
    • Can provide Team Leader with updates on how data collection and analysis is proceeding
    • POC will ensure that data collection for that measure occurs in a timely fashion
step 4 list points of contact poc cont
Step 4: List Points of Contact (POC) (cont.)

Type in Point of Contact for each measure as appropriate.

step 5 list source of data
Step 5: List Source of Data
  • Identify where you will actually get the data for each measure
  • Possible sources of data include:
    • Ordering systems (ADB)
    • Budgeting systems (OROS)
    • Studies that have been conducted (Rate Studies)
    • Hard copy order forms
    • Tally of customer requests
    • Email messages
    • Phone calls
    • Observations
step 5 list source of data cont
Step 5: List Source of Data (cont.)
  • If no known source exists
    • Need to develop methods to collect data
    • Check sheets useful for all kinds of data collection
      • Constructed to whatever size, shape, format appropriate for you
      • Easy to compile data in a way that can be readily graphed
      • Record data for later analysis using bar, pareto, and run charts
      • Provide historical record of the process over time
      • Can be used to introduce data collection to workers who may not be familiar with it
  • May want to use simple check sheet to summarize data already collected but not tallied
example check sheet23
ExampleCheck Sheet

Customer Contacts

July

Aug

Sept

Total

Email

Phone

Visit

Email

Phone

Visit

Email

Phone

Visit

Other ORS Div/Office

1111

1111

1111

12

NIH OD

11

11

1111

8

Clinical Center

111

111

111

111

111

111

18

NCI

1111

1111

1111

111

111

111

111

111

111

30

NHLBI

11

11

11

6

NIAID

--

Other NIH Institute

1

1

1

1

4

HHS

11111111

11111111

11111

21

Outside Agency

11

11

11

11

11

11

12

step 5 list source of data cont24
Step 5: List Source of Data (cont.)
  • For more information on designing data collection forms:
    • Memory Jogger for an example of a Check Sheet (p.31)
    • Statistical Methods for Quality Improvement (pp. 7 - 16)
    • Guide to Quality Control (pp. 30-41)
    • Basic Tools for Process Improvement Module 7: Data Collection (pp. 12-23)
      • http://www.odam.osd.mil/qmo/pdf/datacoll.pdf
    • Process Improvement Notebook for Data Collection Sheet and Check Sheet (pp. 66, 68)
      • http://www.odam.osd.mil/qmo/pdf/pin.pdf
step 5 list source of data cont25
Step 5: List Source of Data (cont.)

Type in Source of Data for each measure you have listed.

step 6 list data collection frequency
Step 6: List Data Collection Frequency
  • Identify the frequency of data collection for each measure
  • Examples include:
    • Ongoing (e.g., ordering systems)
    • Weekly (e.g., count of number of complaints)
    • Monthly (e.g., utilization statistics)
    • Quarterly (e.g., financial reports)
    • Once per fiscal year (e.g., customer survey)
  • Some Common Measures are filled in for you
step 6 list data collection frequency cont
Step 6: List Data Collection Frequency (cont.)

Type in Frequency of Data Collection for each measure.

step 7 list target
Step 7: List Target
  • Identify the target performance level for this FY if possible
  • Targets are usually identified after it is understood what the process is capable of doing
    • Consult Service Level Agreements (SLA) if they exist
    • Examples include:
      • Response time within 3 business days
      • 95% of reports error free
      • Actual asset utilization within 10% of plan
      • Reduce use of vendors by 5%
  • If performance measure is being defined for the first time this year
    • True process capability is being defined for first time
    • Type in “Baseline” under “Target”
  • Targets for some Common Measures are filled in for you
step 7 list target cont
Step 7: List Target (cont.)

Type in Target performance for each measure listed on your Plan.

there are many ways to analyze data
There are Many Ways to Analyze Data
  • Two general types of data
    • Quantitative
    • Qualitative
  • Ways to analyze quantitative data
    • Through visual displays - graphs
    • Through process behavior charts
    • Through statistical analyses
      • Chi-square, t-tests, ANOVA, correlation, regression analyses, factor analysis
    • Through predictive modeling
      • LISREL
common graphs for analysis
Common Graphs for Analysis
  • Pie charts
  • Bar charts
    • Pareto charts
  • Radar charts
  • Line graphs
    • Run charts
    • Process behavior charts
  • Scatter diagrams
    • Correlation
    • Gap analysis
pie charts
Pie Charts
  • Often used to summarize categorical data
  • Show the proportional size of items that make up a whole
  • Convey the relative contribution of different categories to the total
  • Usually used with percentages
  • Good for simple descriptions, quick snapshots of some kinds of data
ors example pie chart
ORS ExamplePie Chart

Conference Services Survey Respondents

N=564

bar graphs
Bar Graphs
  • Useful for comparing different categories by contrasting heights of various bars
  • Helpful in making comparisons among items
    • Frequency, size, importance, satisfaction, dollars, etc.
  • Often used to show comparisons on more than one dimension
    • Categories of products ordered
ors example bar graph
ORS ExampleBar Graph

Conference Services: Scheduling Actions that Occurred

N=564

ors example bar graph37
ORS ExampleBar Graph

Categories of Photography Products Ordered by Year

pareto charts
Pareto Charts
  • Type of bar chart
  • Display bars in descending order
  • Help to focus efforts on areas that offer the greatest potential for improvement
    • Based on the Pareto principle
    • Most problems are due to a minority of categories of causes
  • For more information:
    • The Memory Jogger for more information (p. 95)
    • Building Continual Improvement (pp. 38-44)
    • Statistical Methods for Quality Improvement (pp. 17-24)
    • Basic Tools for Process Improvement Module 8 - Pareto Charts
      • http://www.odam.osd.mil/qmo/pdf/pareto.pdf
    • The Process Improvement Notebook for Pareto Chart of Causes of Quality form (pp. 70-73)
      • http://www.odam.osd.mil/qmo/pdf/pin.pdf
example pareto chart
ExamplePareto Chart

Improvement Ideas Supported by Customers

N=250

radar charts
Radar Charts
  • Used to compare actual values on a series of categories to ideal values
    • Allows comparison among data points
    • Encourages identifying strengths and weaknesses
  • See The Memory Jogger for more information (p. 137-140)
example radar chart
ExampleRadar Chart

Ideal Value

Actual Value

line graphs
Line Graphs
  • Used to study data for patterns
  • Helpful in making comparisons over time
    • Show changes in numerical amounts
    • Identify sequences/changes in data
    • Demonstrate performance before and after an intervention
    • Can be used to predict future performance
  • Run charts and process behavior charts are types of line graphs
  • For more information:
    • The Memory Jogger (pp. 141-144)
    • Building Continual Improvement
    • The Process Improvement Notebook for Run Chart and Control Chart forms (pp. 82-98)
      • http://www.odam.osd.mil/qmo/pdf/pin.pdf
example line graph
ExampleLine Graph

Ratings of Responsiveness to Customer Complaints by Year

N=125

ors example line graph
ORS ExampleLine Graph

NIH ID Cards Issued by Year

scatter diagrams
Scatter Diagrams
  • Demonstrate the relationship between two variables
  • Values on two variables are plotted on a graph to visually show the relationship
  • For more information:
    • The Memory Jogger (pp. 145-149)
    • Statistical Methods for Quality Improvement (pp. 67-89)
    • The Process Improvement Notebook for Scatter Diagram Worksheet (pp. 78-81)
      • http://www.odam.osd.mil/qmo/pdf/pin.pdf
example scatter diagram
ExampleScatter Diagram

Gap Analysis of Customer Ratings of Satisfaction and Importance

Each symbol indicates both the importance and satisfaction rating for a variable, such as timeliness.

analyzing data with graphs
Analyzing Data With Graphs

Customer segmentation charts are good examples of these.

analyzing data with graphs cont
Analyzing Data With Graphs (cont.)

Customer satisfaction results will be provided to Service Groups using radar charts.

analyzing data with graphs cont49
Analyzing Data With Graphs (cont.)

Internal Business Process measures often require run charts or control charts. Control charts are a special type of run chart. Training on process behavior charts will be available (http://www.nih.gov/od/ors/od/oqm/asa/asa_training.htm)

analyzing data with graphs cont50
Analyzing Data With Graphs (cont.)

Scatter charts depict the correlation between 2 variables and can be used to investigate hunches you may have about relationships.

tips for graph analysis
Tips for Graph Analysis
  • Realize that analyzing data is a skill
    • With experience you will get better
    • Analysis is both a science and an art
  • Common methods to interpret graphical data
    • Compare size of categories to each other
    • Compare self to target
    • Compare self to others who are similar
    • Compare self to industry standards
    • Compare self over time
  • General things to look for in data
    • Highlight similarities and differences
    • Identify trends or patterns
    • Notice if anything is missing (e.g., customer groups)
    • Look for themes
tips for graph analysis cont
Tips for Graph Analysis (cont.)
  • Make conclusions based on graphs
    • Summarize what you learned from the data
    • Diagnose problems identified by data
    • Identify any potential solutions suggested by data
  • Generate potential actions based on what you have learned from the data
    • Can you make changes to address what you learned from data?
    • How might you implement those actions?
    • State actions in terms of recommendations
summarizing analyses in your asa presentation
Summarizing Analyses in your ASA Presentation
  • Organize your graphs and conclusions to tell the story of your ASA
    • Who are your customers (i.e., customer segments)
    • Are you customers satisfied with your products/services (i.e., customer satisfaction)?
    • What have you learned about your internal business processes from the process maps and measures?
    • Learning and Growth analyses and conclusions
    • Financial findings regarding unit costs, asset utilization
    • Recommendations for improvement based on data gathered and analyzed
  • ASA Presentation Template and tips will be available this summer via the ASA web page
summarizing analyses in your asa presentation cont
Summarizing Analyses in Your ASA Presentation (cont.)
  • Beware of excess detail
    • Do NOT place all graphs in main section of presentation
      • Select only the most informative graphs for main portion of ASA Presentation
      • Insert narrative slides that include your conclusions of the graphs, findings from the data
      • Place graphs not used in main presentation in Appendices to your main presentation
    • Try to see the “forest through the trees”
    • Look for themes and major findings as a result of the ASA
    • Include recommendations and proposed follow-on actions
      • Make realistic recommendations that could actually be implemented
      • If have power to implement suggested changes, do so
customer segmentation
Customer Segmentation

1. Select customer characteristic(s) to segment

  • NIH IC
  • Location
    • On-campus, Off-campus
    • Building
  • Type of customer
    • NIH employees, contractors, visitors

2. Select product/service dimension

  • New equipment sales
  • Trouble reports
  • Requests for service
  • Frequency of use

3. Determine time frame

  • Current fiscal year, several fiscal years, current month, several months
customer segmentation cont
Customer Segmentation (cont.)

4. Generate chart(s)

5. Review and interpret chart(s)

Which segments are your primary customers?

Is there any pattern to your chart(s)?

Which NIH customers are not currently your customers? Why?

What does the information say about who is and who is not your customer?

pie chart customer segmentation of discrete services
Pie Chart Customer Segmentation of Discrete Services
  • Discrete Service = Manage concession services program, contracts, and use agreements
  • Customer characteristic = Customer Type
  • Product/Service dimension = Frequency of Use
  • Time frame = January through April, 2001
  • Methodology
    • Surveys handed out to all willing participants at point of sale at 8 Dining Halls on campus for one day during the data collection period
pie chart customer segmentation of discrete services cont
Pie Chart Customer Segmentation of Discrete Services (cont.)

1. Collect data and enter in Excel

List Customer Type Horizontally

Label Frequency of Use Measure

Enter Data

pie chart customer segmentation of discrete services cont61
Pie Chart Customer Segmentation of Discrete Services (cont.)

2. Generate Chart

  • Select all labeled cells and data using mouse
  • Choose “Insert”, “Chart”, “Pie”
  • Click “Finish”
pie chart customer segmentation of discrete services cont62
Pie Chart Customer Segmentation of Discrete Services (cont.)

3. Format Chart

  • Click on chart
  • Click right mouse button
  • Choose “Chart Options”
  • Choose “Titles”, “Chart titles”: Type in chart title (e.g., Concession Service Customers)
  • Choose “Data Labels”: Click ”Show percent”
  • Click “OK”
pie chart customer segmentation of discrete services cont63
Pie Chart Customer Segmentation of Discrete Services (cont.)

4. Add important notes to chart

  • Ensure that “Drawing” toolbar is available to you
    • Choose “View”, “Toolbar”, “Drawing”
  • Click on chart
  • Click on “Text Box” from Drawing Toolbar
  • Place text box on chart
  • Type in note and format text

Concession Service Customers

Federal Government

9%

Employees

6%

Fellows/Visiting

Researchers

10%

Contractors

53%

Visitors/

Conference Attendees

22%

Other

Data obtained from 564 customers at all

concessions during January - April, 2001.

bar graph customer segmentation of discrete services
Bar GraphCustomer Segmentation of Discrete Services
  • Discrete Service = Conduct collaborative bioengineering and physical science research
  • Customer characteristic = NIH IC
  • Product/Service dimension = Number of Collaborations
  • Time frame = FY01
  • Data Collection
    • Data collected on all collaborations during FY01
bar graph customer segmentation of discrete services cont

List Customer Type Horizontally

Type Measure Label

Enter Data

Bar Graph Customer Segmentation of Discrete Services (cont.)

1. Collect data and enter in Excel

bar graph customer segmentation of discrete services cont66
Bar Graph Customer Segmentation of Discrete Services (cont.)

2. Generate Chart

  • Select all labeled cells and data using mouse
  • Choose “Insert”, “Chart”, “Column”
  • Click “Finish”
bar graph customer segmentation of discrete services cont67
Bar Graph Customer Segmentation of Discrete Services (cont.)

3. Format Chart

  • Click on chart
  • Click right mouse button
  • Choose “Chart Options”
  • Choose “Titles”, “Chart title”: Type in chart title (e.g., # Collaborations with NIH ICs)
    • Choose “Category (X) Axis”: Type in Customer Type (e.g., NIH IC)
    • Choose “Category (Y) Axis”: Type in Measure Label (e.g., # Collaborations)
  • Choose “Legends”: Click “Show legend”
  • Choose “Data Labels”: Click “Show value”
  • Click “OK”
bar graph customer segmentation of discrete services cont68

# Collaborations With NIH ICs

20

18

16

15

# Collaborations

10

7

5

5

1

0

NCI

NIA

NIAID

NIMH

NINDS

NIH IC

Bar Graph Customer Segmentation of Discrete Services (cont.)

3. Format Chart (cont.)

  • Click on gray area on chart (e.g., plot area)
  • Click right mouse button
  • Choose “Format Plot Area”
  • In the “Area” section, click on the white box.
  • Click “OK”
  • Click on gridlines in center of chart
  • Click right mouse button
  • Choose “Clear”
bar graph customer segmentation of discrete services cont69

# Collaborations With NIH ICs

20

18

16

15

# Collaborations

10

7

5

5

1

0

NCI

NIA

NIAID

NIMH

NINDS

NIH IC

Data based on 47 collaborations in FY01

Bar Graph Customer Segmentation of Discrete Services (cont.)

4. Add important notes to chart

  • Ensure that “Drawing” toolbar is available to you
    • Choose “View”, “Toolbar”, “Drawing”
  • Click on chart
  • Click on “Text Box” from Drawing Toolbar
  • Place text box on chart
  • Type in note and format text
pareto chart customer segmentation of discrete services
Pareto Chart Customer Segmentation of Discrete Services
  • Discrete Service = Conduct collaborative bioengineering and physical science research
  • Customer characteristic = NIH IC
  • Product/Service dimension = Number of Collaborations
  • Time frame = FY01
  • Data Collection
    • Data collected from all DBEPS collaborators on the number of collaborations during FY01

Note: Data is same as bar graph example, but pareto chart will re-order NIH ICs from the greatest to the fewest number of collaborations.

pareto chart customer segmentation of discrete services cont
Pareto Chart Customer Segmentation of Discrete Services (cont.)

1. Collect data and enter in Excel (see slide 65)

  • Click “Data”, “Sort”
  • Ensure correct row (e.g., row with data) is displayed in “Sort By” box. If not, select correct row
  • Click “Descending”
  • Click “OK”

Select IC and Data Cells using Mouse

Data is resorted by IC in descending order

pareto chart customer segmentation of discrete services cont72
Pareto Chart Customer Segmentation of Discrete Services (cont.)

2. Generate Chart

  • See Slide 66

3. Format Chart

  • See Slides 67-68

# Collaborations With NIH ICs

20

18

16

15

# Collaborations

10

7

5

5

1

0

NINDS

NCI

NIAID

NIMH

NIA

NIH IC

Data based on 47 collaborations in FY01

bar graph with added dimension customer segmentation of discrete services
Bar Graph with Added Dimension Customer Segmentation of Discrete Services
  • Discrete Service = Conduct collaborative bioengineering and physical science research
  • Customer characteristic = NIH IC
  • Product/Service dimension = Number of Collaborations
  • Time frame = FY01 and FY02
  • Data Collection
    • Data collected on all collaborations during FY01 and FY02

Added Dimension

bar graph with added dimension customer segmentation of discrete services cont
Bar Graph with Added Dimension Customer Segmentation of Discrete Services (cont.)

1. Collect data and enter in Excel

List Customer Type Horizontally

Type Measure Labels

Enter Data

bar graph with added dimension customer segmentation of discrete services cont75
Bar Graph with Added Dimension Customer Segmentation of Discrete Services (cont.)

2. Generate Chart

  • Select all labeled cells and data using mouse
  • Choose “Insert”, “Chart”, “Column”
  • Click “Finish”
bar graph with added dimension customer segmentation of discrete services cont76
Bar Graph with Added Dimension Customer Segmentation of Discrete Services (cont.)

3. Format Chart

  • Click on chart
  • Click right mouse button
  • Choose “Chart Options”
  • Choose “Titles”, “Chart title”: Type in chart title (e.g., # Collaborations With NIH ICs by Fiscal Year)
    • Choose “Category (X) Axis”: Type in Customer Type (e.g., NIH IC)
    • Choose “Category (Y) Axis”: Type in Measure Label (e.g., # Collaborations)

# Collaborations With NIH ICs by Fiscal Year

20

18

18

18

16

  • Choose “Data Labels”:
  • Click “Show value”
  • Click “OK”

15

12

FY01

# Collaborations

10

8

7

FY02

5

5

3

1

0

NCI

NIA

NIAID

NIMH

NINDS

NIH IC

bar graph with added dimension customer segmentation of discrete services cont77
Bar Graph with Added Dimension Customer Segmentation of Discrete Services (cont.)

3. Format Chart (cont.)

  • Click on gray area on chart (e.g., plot area)
  • Click right mouse button
  • Choose “Format Plot Area”
  • In the “Area” section, click on the white box.
  • Click “OK”
  • Click on gridlines in center of chart
  • Click right mouse button
  • Choose “Clear”

# Collaborations With NIH ICs by Fiscal Year

20

18

18

18

16

15

12

FY01

# Collaborations

10

8

7

FY02

5

5

3

1

0

NCI

NIA

NIAID

NIMH

NINDS

NIH IC

bar graph with added dimension customer segmentation of discrete services cont78
Bar Graph with Added Dimension Customer Segmentation of Discrete Services (cont.)

4. Add important notes to chart

  • Ensure that “Drawing” toolbar is available to you
    • Choose “View”, “Toolbar”, “Drawing”
  • Click on chart
  • Click on “Text Box” from Drawing Toolbar
  • Place text box on chart
  • Type in note and format text

# Collaborations With NIH ICs by Fiscal Year

20

18

18

18

16

15

12

FY01

# Collaborations

10

8

7

FY02

5

5

3

1

0

NCI

NIA

NIAID

NIMH

NINDS

Data based on 47 collaborations in

NIH IC

FY01 and 59 in FY02

customer satisfaction ratings
Customer Satisfaction Ratings

1. Review methodology with OQM before distributing surveys to any customers

2. Completed surveys will be analyzed by OQM and summary charts will be provided

3. Review the radar charts and interpret the findings

  • Compare rating dimensions on each chart
  • What is highest? Why?
  • What is lowest? Why?

4. Review the scatter diagram (gap analysis)

  • What do customers feel is most important?
  • What was their satisfaction on the most important dimensions?
  • What actions can be taken to address the situation?

5. Focus first improvement efforts on areas most important to customers with lower satisfaction ratings

radar chart customer satisfaction ratings
Radar Chart Customer Satisfaction Ratings

Mean Product/Service satisfaction ratings provided for ORS overall on the left and the Service Group on the right.

radar chart customer satisfaction ratings cont
Radar Chart Customer Satisfaction Ratings (cont.)

Mean customer service ratings provided for ORS overall on the left and the Service Group on the right.

scatter diagram gap analysis customer importance and satisfaction ratings
Scatter Diagram: Gap Analysis Customer Importance and Satisfaction Ratings

Mean satisfaction and importance ratings for the Service Group.

internal business process measures
Internal Business Process Measures

1. Identify process measures using deployment process maps, known problem areas, customer feedback

2. Implement methodology to collect data

3. Collect data

4. Generate chart(s)

5. Review and interpret chart(s)

  • What do the process measures tell us?
  • Can we determine why problems are occurring?
  • Is process improving or declining? Why?
  • What can be done to impact process?

6. Identify and report on areas for improvement

run chart internal business process measures
Run ChartInternal Business Process Measures
  • Unique Measure = Reduce number of bills processed with errors
  • Methodology = Count number of bills returned by customer noting an error
  • Point of Contact = Billing Supervisor
  • Source of Data = Log of bills returned classified by customer and type of error
  • Frequency = Quarterly
  • Target = 5% reduction in bills returned
run chart internal business process measures cont
Run ChartInternal Business Process Measures (cont.)

1. Collect data and enter in Excel

List Frequency Intervals Horizontally

Label Data Being Collected

Enter Data

run chart internal business process measures cont88
Run ChartInternal Business Process Measures (cont.)

2. Generate Chart

  • Select all labeled cells and data using mouse
  • Choose “Insert”, “Chart”, “Line”
  • Click “Finish”
run chart internal business process measures cont89
Run ChartInternal Business Process Measures (cont.)

3. Format Chart

  • Click on chart
  • Click right mouse button
  • Choose “Chart Options”
  • Choose “Titles”, “Chart title”: Type in title (e.g., # Bills Processed With Errors by Quarter)
  • Choose “Legend”: Click “Show legend”
  • Choose “Data Table”: Click “Show data table”
  • Click “OK”
run chart internal business process measures cont90
Run ChartInternal Business Process Measures (cont.)

3. Format Chart (cont.)

  • Click on gray area on chart (e.g., plot area)
  • Click right mouse button
  • Choose “Format Plot Area”
  • In the “Area” section, click on the white box.
  • Click “OK”
  • Click on gridlines in center of chart
  • Click right mouse button
  • Choose “Clear”
run chart internal business process measures cont91
Run ChartInternal Business Process Measures (cont.)

4. Add important notes to chart

  • Ensure that “Drawing” toolbar is available to you
    • Choose “View”, “Toolbar”, “Drawing”
  • Click on chart
  • Click on “Text Box” from Drawing Toolbar
  • Place text box on chart
  • Type in note and format text
run chart internal business process measures cont92
Run ChartInternal Business Process Measures (cont.)
  • Run Chart shows that the change in process in Quarter 2 of FY02 appears to have resulted in fewer errors in billing
  • Process behavior charts (i.e., control charts) use formulas to help determine whether the difference is significant
  • Training on process behavior charts will be available. Check the ASA web page (http://www.nih.gov/od/ors/od/oqm/asa/asa_training.htm)
scatter diagram relationship between two measures
Scatter DiagramRelationship Between Two Measures
  • After completing data analyses on other measures, a scatter chart may be generated
    • Further examine relationships between 2 measures
    • Relationship may be suggested by other analyses
    • Example - examine the relationship between two unique measures
      • Unscheduled repairs and age of equipment
  • Graph data and study relationship
    • Positive in nature
    • Negative in nature
scatter diagram relationship between two measures cont
Scatter DiagramRelationship Between Two Measures (cont.)

1. Collect data and enter in Excel

List Categories and Definitions Horizontally

Enter Data

scatter diagram relationship between two measures cont95
Scatter DiagramRelationship Between Two Measures (cont.)

2. Generate Chart

  • Select all data cells using mouse
  • Choose “Insert”, “Chart”, “XY (Scatter)”
  • Click “Next”
  • Choose “Data Range”. Click “Columns”
  • Click “Finish”
scatter diagram relationship between two measures cont96
Scatter DiagramRelationship Between Two Measures (cont.)

3. Format Chart

  • Click on chart
  • Click right mouse button
  • Choose “Chart Options”
  • Choose “Titles”, “Chart title”: Type in chart title (e.g., Repair Calls by Age of Equipment)
    • Choose Value (X) Axis: Type in X axis title (e.g., Type of Repair Call)
    • Choose Value (Y) Axis: Type in Y axis title (e.g., Age of Equipment)
  • Choose “Legend”
  • Click “Show legend”
  • Click “OK”
scatter diagram relationship between two measures cont97
Scatter DiagramRelationship Between Two Measures (cont.)

4. Format Chart (cont.)

  • Click on chart
  • Double click left mouse button on values on X axis
  • Choose “Scale”
    • Choose “Minimum”: Type in 0
    • Choose “Maximum”: Type in 3
    • Choose “Major Unit”: Type in 1
    • Choose “Minor Unit”: Type in 1
scatter chart relationship between two measures cont
Scatter ChartRelationship Between Two Measures (cont.)

4. Format Chart (cont.)

  • Click on gray area on chart (e.g., plot area)
  • Click right mouse button
  • Choose “Format Plot Area”
  • In the “Area” section, click on the white box.
  • Click “OK”
  • Click on gridlines in center of chart
  • Click right mouse button
  • Choose “Clear”
scatter chart relationship between two measures cont99
Scatter ChartRelationship Between Two Measures (cont.)

5. Add important notes to chart

  • Ensure that “Drawing” toolbar is available to you
    • Choose “View”, “Toolbar”, “Drawing”
  • Click on chart
  • Click on “Text Box” from Drawing Toolbar
  • Place text box on chart
  • Type in note and format text
scatter chart relationship between two measures cont100
Scatter ChartRelationship Between Two Measures (cont.)

6. Run Regression Analysis on Data

  • Choose “Tools”, “Data Analysis”, “Regression”
  • Click “OK”
  • Click inside “Input Y Range” box
  • Select all Y axis data cells (e.g., repair data cells)
  • Click inside “Input X Range” box
  • Select all X axis data cells (e.g., age of equipment cells)
  • Click “Output Range”. Click in Output Range box
  • Click on blank cell on data sheet. Click “OK”
scatter chart relationship between two measures cont101
Scatter ChartRelationship Between Two Measures (cont.)

6. Run Regression Analysis on Data (cont.)

  • Correlation = number between 0 (no relationship) and 1 (1 to 1 relationship)
  • Significance level = .05 indicates 95% confidence level

Correlation = .61

Number of Observations = 22

Significance Level = <.002

learning and growth perspective graphing and analyzing turnover sick leave complaints awards

Learning and Growth Perspective Graphing and Analyzing Turnover, Sick Leave, Complaints, Awards

turnover sick leave complaints awards
Turnover, Sick Leave, Complaints, Awards

1. Data is being obtained from ORS HR IT systems

2. OQM will graph data and provide to ASA Teams

3. ASA Teams meet to review and discuss the data

4. Highlight conclusions from the discussion in the final ASA presentation

qualitative analysis of readiness index
Qualitative Analysis of Readiness Index

1. The Readiness Index will be available this summer via the ASA web page

2. ASA Teams meet to review and discuss the questions on the Readiness Index

3. Prepare a short narrative summary of conclusions included as Appendix in final ASA presentation

4. Highlight conclusions in the final ASA presentation

unit cost
Unit Cost

1. Obtain guidance from OBSF on the definition of unit cost measures for your Discrete Services

2. Determine how you can obtain your unit cost data

3. Gather your unit cost data

4. Graph your unit cost data and interpret the findings

  • Are your costs going up? Why?
  • Are your costs going down? Why?
run chart unit cost cont
Run ChartUnit Cost (cont.)

Unit Cost by Quarter

Data might be displayed using a run chart (shown) or control chart.

asset utilization
Asset Utilization

1. Obtain guidance from OBSF on how to define assets for your Discrete Services

2. Determine how you can obtain your planned and actual asset utilization data

3. Gather your data

4. Graph your asset utilization data and interpret the findings

run chart asset utilization
Run ChartAsset Utilization

Data might be displayed using a run chart showing % deviation from plan.

run chart asset utilization112
Run ChartAsset Utilization

1. Calculate the % deviation from plan by quarter

2. Calculate the % deviation from plan cumulatively

run chart asset utilization cont
Run ChartAsset Utilization (cont.)

1. Collect data and enter in Excel

List Data Collection Intervals Horizontally

Label Deviation Measures

Enter Deviation Percentages

run chart asset utilization cont114
Run ChartAsset Utilization (cont.)

2. Generate Chart

  • Select all labeled cells and data using mouse
  • Choose “Insert”, “Chart”, “Custom Types”, “Line - Column on 2 Axes”
  • Click “Finish”
run chart asset utilization cont115
Run ChartAsset Utilization (cont.)

3. Format Chart

  • Click on chart
  • Click right mouse button
  • Choose “Chart Options”
  • Choose “Titles”, “Chart title”: Type in chart title (e.g., Asset Utilization: % Deviation From Plan)
    • Choose “Category (X) Axis”: Type in data collection interval (e.g., Quarter)
    • Choose “Value (Y) Axis”: Type in title (e.g., % Deviation by Quarter (Bar))
    • Choose “Second value (Y) Axis”: Type in title (e.g., Cumulative % Deviation (Line))
  • Choose “Legend”, Click “Show legend”
  • Choose “Data Table”, Click “Show data table”
  • Click “OK”
run chart asset utilization cont117
Run ChartAsset Utilization (cont.)

3. Format Chart (cont.)

  • Click on gray area on chart (e.g., plot area)
  • Click right mouse button
  • Choose “Format Plot Area”
  • In the “Area” section, click on the white box. Click “OK”
  • Double Click left mouse button on left “Y” axis
  • Choose “Scale”
    • In “Minimum” box type -.25
    • In “Maximum” box type .25
    • In “Major unit” box type .05
    • Click “OK”
  • Repeat for right “Y” axis
run chart asset utilization cont118
Run ChartAsset Utilization (cont.)

O% = perfect planning (no deviation)

Bars show deviation each quarter

Solid line shows cumulative deviation

unique measures
Unique Measures
  • Use any of the graph types already discussed as appropriate for unique measures you have added
    • Refer to slides 47-50 to select the type of graph best suited to the data
    • Try other graph types to see if different displays highlight different results
  • Interpret the data
  • Identify potential actions to address problems areas, issues, improvement opportunities
  • Make recommendations for improvement and plan actions accordingly
  • Include in Final ASA Presentation
    • In main part of presentation if major finding
    • In Appendices if back-up material
summary
Summary
  • Data collection can be time consuming
    • PLAN before you act
    • Each ASA Team should complete a Data Collection Plan
  • Analyzing data is both a science and an art
    • Use graphs to summarize data
      • Pie, bar, pareto, radar, line, run, scatter
      • ASA Team review of the graphs is basis of the data analysis and interpretation
  • Organize your graphs and conclusions to tell the story of your ASA
    • Look for themes and major findings
    • Identify potential actions to address problems areas, issues, improvement opportunities
    • Make recommendations for improvement and plan actions accordingly
resources
Resources
  • Brassard, M., & Ritter, D. (1994). The memory jogger. Salem, NH: GOAL/QPC.
  • Culbertson, A., Houston, A., Faast, D., White, M., Aguirre, M., & Behr, C. (1997). The Process Improvement Notebook (TQL 97-01). Washington, DC: Department of the Navy. http://www.odam.osd.mil/qmo/pdf/pin.pdf
  • Ishikawa, K. 1982. Guide to quality control. Tokyo, Japan: Asian Productivity Organization.
  • Kume, H. (1989). Statistical methods for quality improvement. Tokyo, Japan: Association for Overseas Technical Scholarship.
  • Navy Total Quality Leadership Office. (1996). Basic Tools for Process Improvement. Washington, DC: Department of the Navy. http://www.odam.osd.mil/qmo/pdf/pareto.pdf
  • Wheeler, D. J. (2000). Understanding variation: The key to managing chaos. Knoxville, TN: SPC Press.
  • Wheeler, D. J., & Poling, S. R. (1998). Building continual improvement. Knoxville, TN: SPC Press.