Use of Growth Models to Measure
Download
1 / 25

Use of Growth Models to Measure Adequate Yearly Progress AYP Under the No Child Left Behind NCLB Act - PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on

Use of Growth Models to Measure Adequate Yearly Progress (AYP) Under the No Child Left Behind (NCLB) Act . Lou Jacobson Presentation at the IES Research Conference June 7, 2007. Topics to be Covered. Research for REL Appalachia:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Use of Growth Models to Measure Adequate Yearly Progress AYP Under the No Child Left Behind NCLB Act' - rune


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Use of Growth Models to Measure Adequate Yearly Progress (AYP) Under the No Child Left Behind (NCLB) Act

Lou Jacobson Presentation at the

IES Research Conference

June 7, 2007


Topics to be covered
Topics to be Covered

Research for REL Appalachia:

  • Year-1 study examining the likelihood Virginia schools would have all students proficient in reading & math by 2014.

    • Today’s focus is on reading results for all students together.

    • The study also covers math and each of 6 subgroups.

    • The report will be on the REL website in about a month.

  • Year-2 study examines a second key NCLB goal—identifying under-performing schools

    • Use growth models developed in year-1.

    • Examine the accountability systems in Virginia, Kentucky, and Tennessee using school level data.


Virginia s nclb status standard
Virginia’s NCLB Status Standard

  • 100% of the students in a school must be proficient in math and reading by 2014

  • Adequate Yearly Progress (AYP) requires meeting interim standards annually.



As proficiency levels rise how will growth change
As proficiency levels rise, how will growth change?

Strategy for answering this question:

  • Estimate the relationship between the 2002 proficiency level and the change in proficiency 2002-05.

  • Forecast growth assuming that as proficiency rises schools starting at low levels will follow the same path as schools starting out at high levels.



Our growth model
Our Growth Model

ΔP= α+ β P0 + ε

where:

ΔP = (P3 – P0)/3 (annualized proficiency change)

Ps,t is the percent of test-takers scoring

proficient in school “s” in year “t”

α = the intercept coefficient

β = the slope coefficient

ε = error term

Averaging the proficiency change over 3 years reduces the effect of random variations, which accounts for as much as 75 percent of year-to-year change.


Model estimates
Model Estimates

Standard errors in parentheses


What does the model say about high schools
What Does the Model Say about High Schools?

  • Annual proficiency growth fell by 2.2 percentage points for each 10-point gain in proficiency.

  • It becomes progressively more difficult to boost proficiency as higher and higher proficiency levels are reached.

  • Schools are unlikely to raise proficiency by a constant number of students as the pool of non-proficient students shrinks.


How high will proficiency rise
How High Will Proficiency Rise?

  • Is there a proficiency ceiling?

    • VA schools have had major increases in proficiency, BUT

    • Proficiency levels will plateau in about 3 years

  • How do we know? Solve the growth equation for zero growth.

    C = α / β

    where:

    C = the proficiency ceiling

    α = the intercept coefficient

    β = the slope coefficient

  • Where are the plateaus?


Steady state proficiency levels
Steady State Proficiency Levels

13 points

20 points

27 points

Upper Limit

Lower limit = 1 std dev below average


Implications of these findings
Implications of These Findings

  • Few, if any, schools will sustain 100% proficiency levels, if current patterns continue.

  • Schools under-performing the status standard can avoid being labeled “in need of improvement” because there is an alternative NCLB standard.

  • Safe Harbor (Growth) Standard requires that schools not meeting the status standard must reduce the percentage of non-proficient students by 10 percent from one year to the next.

    • A school at 80 percent proficiency has to increase proficiency by 2 percentage points [100-80 = 20; 10% of 20 = 2]

    • A school at 90 percent proficiency has to increase proficiency by 1 percentage point.

  • In contrast, the Virginia Status Standard rises by 4 points per year.


Can a growth standard identify under performers
Can a growth standard identify under-performers?

  • Yes, if we can determine how proficiency changes among schools facing similar challenges.

  • To do this it is necessary to develop a peer-group standard—the standard experts agree provides the best means to identify schools needing improvement.


Developing a peer group standard
Developing a Peer-Group Standard

  • Use our growth model to measure the difference between actual and predicted growth over 3-years.

  • Identify under-performing schools based on performance being below average by a statistically significant amount.

  • Group schools based on exogenous characteristics such as students’ entry level proficiency.


Distribution of High Schools Sorted by Difference

Between Actual and Predicted Performance

Under performing

Over performing


What the figure tells us
What the figure tells us

  • The difference between actual and predicted growth is close to normally distributed.

  • Most schools’ actual performance is within one standard deviation of predicted levels.

  • A few schools under-perform by large and statistically significant amounts.

  • A few schools over-perform by large and statistically significant amounts.


Advantages of using the growth model
Advantages of Using the Growth Model

  • The growth model provides:

    • A “rating” for each school that tells us how far its performance is from average for schools with similar baseline proficiency.

    • Measures of the statistical significance of the ratings.

  • In contrast, Virginia’s NCLB AYP standards provide:

    • Indiscriminant pass/fail measures.

    • Ambiguous indicators of whether performance is truly below a standard.


Benefits of the peer group concept
Benefits of the Peer-Group Concept

  • Improvement efforts can be focused on schools needing improvement that should be able to make substantial progress.

  • Incentives to improve performance will be strengthened by not asking educators to do the impossible.

  • External pressure to correct problems will be increased because confidence that problems are real and can be effectively addressed will grow.


Benefits of identifying over performing schools
Benefits of Identifying Over-Performing Schools

  • Comparisons between over-performing schools and under-performing schools can identify factors that are:

    • not under the control of school officials that should be taken into account in creating peer-groups.

    • under the control of school officials that make a substantial difference among schools facing similar challenges.


Can safe harbor standards identify under performers
Can Safe Harbor Standards Identify Under-Performers?

Yes, if:

  • Growth is averaged over several years.

  • Tests of the statistical significance are applied.

    However, regression-based standards better measure average performance of schools at different proficiency levels.


Some Schools are Misclassified by Current Standards

Under performing

Over performing

Some of

these schools

miss AYP

Some of these schools

make AYP




Summary proficiency forecasting
Summary: Proficiency Forecasting Performance Quintile

  • The simple linear model using base-period proficiency fits the data well and produces reasonably small confidence intervals.

  • Additional factors are unlikely to have much of an effect.

  • The future may not precisely track the past, but is likely to come reasonably close.

  • It would be useful to see how well our model works in projecting proficiency in other states.


Summary peer group modeling
Summary: Peer-Group Modeling Performance Quintile

  • Our peer-group model comes reasonably close to:

    • Revealing how far above or below average is actual performance.

    • Identifying cases where deficits are statistically significant.

  • Averaging growth over several years greatly improves the statistical reliability of the measures.

  • Our model addresses widely recognized problems that reduce support for the positive aspects of NCLB.


ad