Use of confidence intervals in performance improvement appeals
Download
1 / 14

use of confidence intervals in performance improvement appeals - PowerPoint PPT Presentation


  • 200 Views
  • Uploaded on

Use of Confidence Intervals in Performance Improvement Appeals. Performance Reporting Division Texas Education Agency. Overview of CI Process. Eligibility for application of CI Matched pairs of students Recalculate performance improvement Recalculate required improvement

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'use of confidence intervals in performance improvement appeals' - betty_james


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Use of confidence intervals in performance improvement appeals l.jpg

Use of Confidence Intervals in Performance Improvement Appeals

Performance Reporting Division

Texas Education Agency


Overview of ci process l.jpg
Overview of CI Process Appeals

  • Eligibility for application of CI

  • Matched pairs of students

  • Recalculate performance improvement

  • Recalculate required improvement

  • Calculate upper limit of CI

  • Compare upper limit to recalculated RI


Eligibility for application of ci l.jpg
Eligibility for Application of CI Appeals

  • To be eligible to have confidence intervals applied to performance improvement on appeal, the district/campus must have shown improvement on:

    • The measure in question, and

    • The other measure (graduation or attendance for the specific student group).

  • Appeals for CI by ineligible districts/campuses were denied.


Matched pairs of students l.jpg
Matched Pairs of Students Appeals

  • Next we used the student level data to create matched pairs of students.

  • Students in 2004 were matched by student group and test (TAKS vs. SDAA, etc.) to students in 2005.

  • Unmatched students were dropped from the analysis.


Recalculate performance rates l.jpg
Recalculate Performance Rates Appeals

  • Passing rate of the matched pairs of students in 2004 (ppy).

  • Passing rate of the matched pairs in 2005 (pcy).

  • A third passing rate, the passing rate of matched pairs across both years (ppc) was also calculated, which will come into play later.


Recalculate performance improvement l.jpg
Recalculate Performance Improvement Appeals

  • Performance improvement was calculated as the difference between the passing rate of the matched pairs of students in 2004 (ppy) and the passing rate of the matched pairs in 2005 (pcy).


Recalculate required improvement l.jpg
Recalculate Required Improvement Appeals

  • Required improvement was calculated by plugging ppy (matched pair performance in 2004) into the RI formula on p. 26 of the 2005 AYP Guide.

    RI = 100 – ppy

    10


Calculate confidence interval l.jpg
Calculate Confidence Interval Appeals

  • A 68% one-tailed (upper limit only) Wald confidence interval for performance improvement was computed.

    UL = PI + [(z/sqrt(n))*sqrt((ppy*(1-ppy))+(pcy*(1-pcy))+(2*((ppy*pcy)-ppc))]

    Where:

    UL = Upper limit of the Wald confidence interval

    PI = recalculated performance improvement rate (pcy - ppy)

    z = z-score for selected confidence level (for a 68% CI, z = 0.47)

    n = number of matched pairs

    ppy = performance (passing rate) of matched pairs in prior year

    pcy = performance (passing rate) of matched pairs in current year

    ppc = performance (passing rate) of matched pairs in prior year and current year


Slide9 l.jpg

We generate a range of potential PI values based on the recalculated PI and its standard error to get a normal curve.


Slide10 l.jpg

Calculating the upper limit using the Wald formula gives us a range (or interval) of PI values we are confident the true PI value falls in (the green area).

Upper Limit


Compare upper limit to ri l.jpg
Compare Upper Limit to RI a range (or interval) of PI values we are confident the true PI value falls in (the green area).

  • The upper limit of the confidence interval must be greater than or equal to required improvement for the appeal to be granted.

  • Another way to say the above is that required improvement must fall inside the confidence interval of performance improvement for the appeal to be granted.


Compare upper limit to ri12 l.jpg
Compare Upper Limit to RI a range (or interval) of PI values we are confident the true PI value falls in (the green area).

If RI falls within CI (green area) then appeal for CI is granted!

If RI falls outside the CI, then appeal is denied.

NOTE: Because RI will never be less than zero, the real area of interest is the portion of the green area between 0 and the UL.


Example from an actual appeal l.jpg

2005 Preliminary AYP a range (or interval) of PI values we are confident the true PI value falls in (the green area).

(Reading, Special Education group)

2005 Met Standard = 49%

2004 Met Standard = 44%

Change (PI) = 5%

RI = 6%

0.1% improvement on other measure

FAILS SAFE HARBOR

Example from an Actual Appeal


Example from an actual appeal cont l.jpg
Example from an Actual Appeal (cont.) a range (or interval) of PI values we are confident the true PI value falls in (the green area).

On Appeal

(Matched pair data)

  • Pcy = 42%

  • Ppy = 22%

  • Ppc = 12%

  • Recalc. PI = 20%

  • Recalc. RI = 8%

  • 68% Wald UL = 24%

    GRANTED

    NOW MEETS AYP


ad