Module 3 using statistics for benchmarking and comparison
Download
1 / 13

Module 3: Using Statistics for Benchmarking and Comparison - PowerPoint PPT Presentation


  • 191 Views
  • Uploaded on

Module 3: Using Statistics for Benchmarking and Comparison. Common Measures Training Chelmsford, MA September 28, 2006. Comparison: Two Questions. Comparing two samples of one population, over time Are we confident performance changed? How much did performance change?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Module 3: Using Statistics for Benchmarking and Comparison' - sauda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Module 3 using statistics for benchmarking and comparison l.jpg

Module 3: Using Statistics for Benchmarking and Comparison

Common Measures Training

Chelmsford, MA

September 28, 2006


Comparison two questions l.jpg
Comparison: Two Questions

  • Comparing two samples of one population, over time

    • Are we confident performance changed?

    • How much did performance change?

  • Comparing two samples from two different states

    • Are we confident performance is different?

    • How much difference is there?


Key concepts for comparison l.jpg
Key Concepts for Comparison

  • "Statistical Significance" /

    Significant Difference

  • Confidence Interval of a Difference

  • Minimum Detectable Difference


Statistical significance l.jpg
“Statistical Significance”

  • Important ERP Question: Did facility performance change between the baseline and the post-certification inspections?

  • Population improvement? If we observed an improvement between the first sample and the second sample, are we confident there was an improvement in the overall population?

    • Remember, both the first and second samples’ point estimates have error associated with them


Statistical significance5 l.jpg
“Statistical Significance”

  • “Statistical significance” means we are confident there was an improvement in the population

  • Other differences. Concept also applies when comparing results from two states

    • Are they significantly different?


Understanding differences l.jpg
Understanding Differences

  • Difference = proportion or quantity. E.g.,

    • Proportion:

      • 40% compliance in Round 1

      • 60% compliance in Round 2

      • Difference = 20%

    • Mean:

      • 300 pounds average waste generation in State A

      • 400 pounds average waste generation in State B

      • Difference = 100 pounds

  • Outcome measures, indicator scores and certification accuracy can have differences, too


Confidence intervals for differences l.jpg
Confidence Intervals for Differences

  • Just like single sample observations, observed differences have error associated with them

  • Confidence interval reflects that error. E.g.,

    • We are 95% confident that compliance increased 20% (+/-7%)

    • We are 95% confident that average waste generation is 75-125 pounds lower in State A than in State B


Is the difference significant l.jpg
Is the Difference Significant?

Two ways to understand if the difference is significant

  • Formal hypothesis test

    • Less flexible

    • Test must be defined pre-sample

  • Or confidence interval of the difference…


Confidence interval of the difference l.jpg
Confidence Interval of the Difference

Confidence interval of the difference contains zero?

  • Difference is 10% +/- 7% (95% conf.)

    • 95% confidence that difference is between 3% and 17% (excludes zero)

    • Difference is statistically significant

  • Difference is 10% +/- 11% (95% conf.)

    • 95% confidence that difference is between -1% and 21% (could be zero)

    • Difference is not statistically significant


Minimum detectable difference l.jpg
Minimum Detectable Difference

= How large a change in performance you need to see for it to be “statistically significant”

  • Important concept in sample planning

  • Depends on population size and sample size for each sample

  • Smaller detectable difference = larger sample size


Minimum detectable difference11 l.jpg
Minimum Detectable Difference

  • Think about what level of change would drive decision-making

    • Worthwhile to design samples to detect a 2% change?

    • Is it different for practical purposes?

    • Would you do anything differently than if you thought there was no change?


Tour of the spreadsheet tools l.jpg
Tour of the Spreadsheet Tools

Two-sample sheets in Sample Planner and Results Analyzer.


For more information l.jpg
For more information…

Contact Michael Crow


ad