1 / 56

Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008

Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008. Presentation by Johnny Wilson, Charles Lanier, and Mike Gallagher, all from the Accountability Services Dept., NCDPI. Purposes – Overall Session.

art
Download Presentation

Effect Size Lens on Local SAT and State Math Test Results NC Accountability Conference February 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Effect Size Lens on Local SAT and State Math Test Results NCAccountability ConferenceFebruary 2008 • Presentation by Johnny Wilson, Charles Lanier, and Mike Gallagher, all from the Accountability Services Dept., NCDPI

  2. Purposes – Overall Session • Provide background and access paths to important data. (important = helpful) • Show how LEAs and schools can access and compare test results across EOG, EOC and SAT. • Entice folks to exploit the usefulness of such comparisons.

  3. Who’s on First? • Johnny Wilson, Reporting Section, SAT Data Report. jwilson@dpi.state.nc.us • Charles Lanier, Test Development Section, State Testing Data in the Green Book. clanier@dpi.state.nc.us • Mike Gallagher, Test Development, Effect Size Look at SAT and State Test Results. mgallagher@dpi.state.nc.us

  4. Purposes – Part 1: SAT • Get acquainted with the major trends in SAT results for North Carolina. • Look at school level variability of SAT results from year-to-year. • Show the access path for information at the school and LEA levels.

  5. SAT Math Score Trends Between 1997 and 2007, the Math gap between NC and the Nation narrowed from 23 pts. to 6 pts. NC’s Math scores fell in 2007 after more than a decade of gains.

  6. SAT Math Score Trends

  7. SAT Verbal Score Trends By comparison, NC’s Verbal scores have been up and down over the same period. Between 1997 and 2007, the Math gap between NC and the Nation has narrowed from 15 pts. to 7 pts.

  8. SAT Verbal Score Trends

  9. SAT Total Score Trends for NC and the Nation • After 10 consecutive years of gains, • NC’s mean total SAT scores fell in • 2006 and 2007. • Gap between NC’s mean total score • and the nation’s score has narrowed • from 53 in 1990 to 13 points in • 2007.

  10. Total Score Trends for NC and the Nation

  11. NC’s Schools with Declining SAT Scores At the school level, math scores also declined in 2006 & 2007, compared with the previous year. Notably, fewer schools increased math scores in 2007 than in 2006.

  12. Percent of NC’s Schools with Declining SAT Scores

  13. Changes in SAT Math Scores Most changes in SAT scores from year to year are not unusual. Low-Volume Schools tend to have larger changes. The larger the test-taking population, the smaller changes in SAT scores tend to be.

  14. Changes in SAT Math Scores by Test-taking Volume (US)

  15. Changes in SAT Math Scores by Test-taking Volume (NC)

  16. Changes in SAT Math Scores All Schools (US)

  17. Changes in SAT Math Scores All Schools (NC)

  18. Factors that Might Influence Score Fluctuations Fatigue Factor SAT fee increase in 2005 resulted in decline in repeat test-takers. Difficulty of the test

  19. NC’s Asian students had the highest SAT score among racial/ethnic groups, followed by White students. Only Asian and White students scored above the state average. NC SAT Math Scores by Race

  20. NC SAT Math Scores by Race Mean SAT Score

  21. SAT Mathematics Scores By Race/Gender Mean SAT Score

  22. For More Information . . . SAT Reports from 1995 to 2007 include information and data for: the Nation the State 117 Public School Systems Charter Schools North Carolina School of the Arts North Carolina School of Science and Mathematics http://www.ncpublicschools.org/accountability/reporting/sat/

  23. Sample SAT Report Page • Some questions to explore: • Are districts closer to the state average in Math than in Critical Reading? • How about variability among schools within districts? • How about participation rates?

  24. Question of the Day … • SAT puts an “official” question of the day on its web site. • Excellent way to get a better understanding of what students will be facing. Why not have 8th graders try them? • www.collegeboard.com

  25. Purposes – Part 2: Green Book • Find out more about getting data from the Green Book, … • both in its standard form … • and its interactive, electronic form.

  26. Electronic GreenBook:A Web-tool for Data

  27. Electronic GreenBook:A Web-tool for Data Your

  28. Electronic GreenBook

  29. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools • Equally accurate • Equally reliable • More timely • More accessible

  30. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  31. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  32. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  33. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  34. year students subject grade geography display Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  35. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools 1. participation 2. performance

  36. year geography grade & subject students Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  37. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools So far … • Questions? • Answers? • 5% / 95% Rule

  38. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  39. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  40. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  41. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  42. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  43. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  44. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  45. Electronic GreenBookhttp://report.ncsu.edu/ncpublicschools

  46. Classic Green Book • SAMPLE PAGES FOLLOWING ….. • Finding average scale scores for LEAs. • Finding standard deviations for EOC and EOG scale scores. • ….. Which leads in to … PART 3

  47. Purposes – Part 3: THE LENS • Discuss effect size, a useful tool for a fuller understanding of LEA or school performance across grade and time. • Compute some math effect sizes and highlight where the information is obtained. • Show the relationship between LEA math performance on SAT, EOC, and EOG.

  48. The Importance of “Where.” • ‘A school (or LEA) is data driven wise if everyone knows where they are and where they are going.” • … Donna Snodgrass, et al.

  49. Where is this LEA? • Math ABC Results for 2006-2007 • (Percent at or above grade level) • EOG Grade 5 ….. 59.8 (state 66.7) • EOG Grade 8 ….. 48.9 (state 65.1) • EOC Geometry …45.0 (state 63.0) • Pass rates and gaps in pass rates leave out a big part of the picture.

  50. The Average is above Average • The percent above a cut score counts a score the same whether it is just above (below) the cut or far above (below) the cut. BUT the average includes all the score information. • Year-to-year changes in percent above are affected by where on the distribution the cut is located. BUT the average follows the center of scores.

More Related