2007 accountability annual meeting n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
2007 Accountability Annual Meeting PowerPoint Presentation
Download Presentation
2007 Accountability Annual Meeting

Loading in 2 Seconds...

play fullscreen
1 / 76

2007 Accountability Annual Meeting - PowerPoint PPT Presentation


  • 151 Views
  • Uploaded on

2007 Accountability Annual Meeting. Evaluation and Reporting Office Florida Department of Education September 6, 2007. Agenda. Welcome and Introductions Responsibilities for Evaluation and Reporting 2006-07 School Grade Changes, Results, and Outlook

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '2007 Accountability Annual Meeting' - betty_james


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
2007 accountability annual meeting

2007 Accountability Annual Meeting

Evaluation and Reporting Office

Florida Department of Education

September 6, 2007

agenda
Agenda
  • Welcome and Introductions
  • Responsibilities for Evaluation and Reporting
  • 2006-07 School Grade Changes, Results, and Outlook
  • 2006-07 AYP Results, Benchmarks, and Safe Harbor
  • Impact of Growth Model
  • AYP in 2007 and Beyond
  • SV23 Matching/Updating Process
  • New Website
  • Existing Website Improvements
  • School Grade Issues for 2007, Going Forward
  • School Improvement Ratings for Alternative Schools Rule Development Workshop
evaluation and reporting staff juan c copa bureau chief
Evaluation and Reporting Staff Juan C. Copa, Bureau Chief
  • Accountability Programs – Ed Croft, Director
    • State Accountability Programs – Tracy Selman, Program Director
      • Analysts – Sue Klos; Gambhir Shrestha, Ph. D.
    • Federal Accountability Programs – Reneé Bruno, Program Director
      • Analyst – Kiersten Farmer
    • Support Staff – Toye Coxe, Staff Assistant
    • Phone – 850-245-0411
    • Email – evalnrpt@fldoe.org
evaluation and reporting staff cont
Evaluation and Reporting Staff (cont.)
  • Research and Evaluation – Marcus Mauldin, Ph. D, Director
    • Analysts – Martha Miller, Ph. D; Jennifer Blalock
  • Support Staff – Tria Parsons, Administrative Secretary
  • Phone – 850-245-0429
  • Email – evalnrpt@fldoe.org
school grades and ayp
School Grades and AYP
  • School Grades Under A+ Plan
    • School grades were first issued in 1999.
    • Since 1999 there have been many changes to the calculation of school grades.
    • We continue to look for ways to improve the calculation.
  • Adequate Yearly Progress (AYP)
    • The federal accountability indicator in the No Child Left Behind Act of 2001.
    • AYP has been calculated since 2003.
    • The growth model was approved in 2007.
schools in need of improvement
Schools in Need of Improvement
  • Required by No Child Left Behind (NCLB).
  • Evaluation and Reporting works with the Division of K-12 Public Schools to determine Schools in Need of Improvement.
  • Only Title I schools subject to sanctions.
alternative schools accountability
Alternative Schools Accountability
  • Part of the A++ Legislation passed during the 2006 Legislative Session (s. 1008.341, F.S.).
  • Provides Alternative Schools the option of receiving a traditional school grade or a school improvement rating.
  • If an alternative school chooses to receive a school improvement rating, the performance of the alternative school’s students will be included in the calculation of the rating and the school grade of the students’ home school.
school improvement ratings for alternative schools 2006 and 2007
School Improvement Ratings for Alternative Schools – 2006 and 2007
  • Alternative Schools were identified based on their Primary Service Type as reported on the Master School Identification (MSID) file and district input and adjustment.
  • Those schools were given the option of receiving a “Points Only” designation or a school grade.
    • “Points Only” refers to the calculation of a school grade without the assignment of a letter grade.
school improvement ratings for alternative schools activities 2006 07
School Improvement Ratings for Alternative Schools – Activities (2006-07)
  • Collected information to identify students who are statutorily to be excluded from the calculation
  • Collected information on a student’s zoned school and district (element description adjusted for 2007-2008 data collection)
  • Held rule development workshops on March 20 and August 30 in Tallahassee
    • Workshop Scheduled for this afternoon (3:30-5:30pm)
school improvement ratings for alternative schools timeline
School Improvement Ratings for Alternative Schools – Timeline
  • April 2007 – Provided districts the opportunity to verify the list of alternative schools in their district (as identified in 2005-06)
  • May 2007 – Provided alternative schools the option of a “Points Only” designation or a school grade
  • August - September 2007 – Conduct rule development workshops on the calculation of school improvement ratings
  • October/November 2007 – Final passage of rule before State Board of Education
  • June 2008 – Release school grades, school improvement ratings for alternative schools, and AYP
summary of issues raised at workshops
Summary of Issues Raised at Workshops
  • Major concerns raised
    • Potential exclusion of choice option schools from the definition of “alternative school”
      • Some charters serve at-risk populations and have students referred to the school
    • Crediting of students back to the “home school”
      • What limitations? Only referred students? Can middle school student performance be credited back to an elementary school?
    • Lack of a full-year enrollment requirement for the school improvement rating
      • Statute specifies Survey 2 and/or Survey 3
calculation issues to be determined
Calculation Issues to be Determined
  • Crediting of students back to the “home school”
    • Limit to only those students referred?
    • Limit to only those students whose “home schools” reflect the same grade configuration as the alternative school?
  • Magnitude of Learning Gains needed to define the categories of improvement – Improving, Maintaining, Declining
    • Improvement of at least 1%, 5%, 10% over the previous performance?
    • Examine other performance measures (e.g., QA for DJJ schools) and determine their compliance with the statutory requirements.
  • Cell Size
    • Unlike school grades, statute outlines that students enrolled in either membership count (fall and/or spring) are to be included.
    • At least 30 students in either count? More than 10 students?
voluntary pre kindergarten vpk accountability
Voluntary Pre-Kindergarten (VPK) Accountability
  • Evaluation and Reporting worked with the Department’s Office of Early Learning to calculate Readiness Rates for VPK Providers
  • Readiness Rates can be found at https://vpk.fldoe.org/
merit award program map
Merit Award Program (MAP)
  • Evaluation and Reporting developed Value Tables that districts may use to measure improved student achievement for teachers in performance pay calculations.
  • Value Tables were provided to measure performance in Reading and Math in Grades 4-10.
    • FCAT
    • NRT (Grades 4 and 5)
  • A total of 19 Value Tables were made available to districts (grade-level and school type-level).
  • Value Table Points were also attached to student records and made available through the INDV file.
supplemental educational services accountability
Supplemental Educational Services Accountability
  • Evaluation and Reporting is working with the Division of K-12 Public Schools to develop an accountability calculation for effectiveness of SES providers.
research analysis and evaluation
Research, Analysis and Evaluation
  • Research, Analysis, and Evaluation.
    • Evaluation and Reporting is tasked with a number of analyses, focusing mainly on examining the impact of programs and policies on student achievement.
sat act psat plan teacher projections and projected high school graduates
SAT/ACT/PSAT/Plan, Teacher Projections and Projected High School Graduates
  • SAT/ ACT/ PSAT/ PLAN and Teacher Projections
    • Multiple results and trends are published on our website.
  • Teacher Projections
    • Multiple trends and statistical reports pertaining to teacher data are on our website.
      • New hires
      • Critical teacher shortage areas
      • Florida teacher retention
      • Supply of New Teachers
  • Projected High School Graduates
2006 07 school grade changes recap
2006-07 School Grade Changes (Recap)
  • Writing
    • Remained at 3.5, no increase to 4.0.
    • Essay only.
  • Science
    • Added as a seventh component.
  • Math, Lowest 25 Percent
    • Added as an eighth component.
  • Revised School Grading Scale
    • 800 point scale changed from current 600 point scale.
  • Retakes of Grade 11 and 12
    • High Schools earned ten bonus points when half of all 11th and 12th graders retaking the FCAT met the graduation requirements in reading and math.
raising standards and higher accountability lead to increased student performance

Red bars indicate years when accountability requirements were increased.

The count of "F" schools drops the year after each major increase in requirements.

140

Lowest Performing Schools

120

100

83

78

78

80

64

49

60

35

40

21

20

4

0

0

1999

2000

2001

2002

2003

2004

2005

2006

2007

Raising Standards and Higher Accountability Lead to Increased Student Performance
school grades 2007 what is measured
School Grades 2007: What is Measured
  • Additional requirements applied AFTER a school’s points are calculated:
  • Adequate Progress for Lowest 25%—required to earn grade based on calculated points. If a school does not meet this requirement, the school’s grade is lowered one letter grade.
  • Participation Requirement (Percent Tested)—required to earn grade based on calculated points. Schools must test at least 95% of their students to earn an “A”, at least 90% to earn any other grade.
students included in performance components
Students Included in Performance Components

Students who are enrolled all year (i.e., enrolled in the fall and spring terms through the testing period) and who are

standard curriculum students*

* Including speech-impaired, gifted, hospital homebound, and English language learners in an ESOL program two or more years.

performance component criterion
Performance Component Criterion:

Are students achieving at or above grade level?

meeting performance criteria
Meeting Performance Criteria:

Grade level performance for math, reading, and science = FCAT achievement level 3;

Grade level performance for writing (essay) = FCAT score of 3.5;

One point is awarded for each percent of students scoring at or above grade level.

performance components points 2007
Performance Components Points 2007

Schools receive one point for each percent of students meeting the performance criteria.

students included in learning gains components
Students Included in Learning Gains Components

All students (including students with disabilities and ELL students) who are enrolled all year (i.e., enrolled in the fall and spring terms through the testing period) and who have both current and prior-year FCAT scores.

learning gains criterion
Learning Gains Criterion:

Are students making at least one year’s worth of progress in a year’s time?

Schools receive one point for each percent of students making learning gains.

meeting the learning gains criterion
Meeting the Learning Gains Criterion:

a. Improving by one or more FCAT achievement levels – e.g., from 1-2, 2-3,

3-4, 4-5;

b. Maintaining FCAT achievement levels 3, 4, or 5;

c. For non-retained students at achievement levels 1 or 2: showing more than one year’s growth on FCAT developmental scale scores (DSS).

students included in learning gains for the lowest performing 25
Students Included in Learning Gains for the Lowest Performing 25%

Of students included in the learning gains calculation, the count of the lowest 25% is based on prior-year FCAT developmental scale scores.

learning gains for the lowest 25 who is included
Learning Gains for the Lowest 25%: Who Is Included

The count of the lowest performing 25% is restricted to students at FCAT achievement levels 1, 2, and 3.

2007 school grades points components total
2007 School Grades Points Components, Total

PLUS 11th and 12th grade retakes for possible bonus points (10)

additional criteria
Additional Criteria
  • Percent Tested (Participation Requirement)
  • Adequate Progress of Lowest Performing Students
percent tested
Percent Tested
  • What percentage of eligible students who should have been tested on the FCAT were actually tested?
  • Grade A schools - 95% or more tested
  • All other grades - at least 90% tested
  • Grade I schools - fewer than 90% tested
adequate progress for the lowest 25 of students
Adequate Progress for the Lowest 25 % of Students
  • Criterion: At least half (50% or more) of the lowest performing students must show learning gains in reading and math.
    • Penalty for missing adequate progress = drop one letter grade.
impact of adequate progress requirement
Impact of Adequate Progress Requirement
  • In 2007, a total of 136 Schools dropped a letter grade due to the Adequate Progress requirement:
    • 66 schools dropped from an “A” to a “B”
    • 22 schools dropped from a “B” to a “C”
    • 48 schools dropped from a “C” to a “D”
  • Impact by School Type
    • 28 elementary schools dropped a letter grade (2% of all elementary schools)
    • 2 middle schools dropped a letter grade (less than 1% of all middle schools)
    • 101 High schools dropped a letter grade (26% of all high schools
    • 5 combination schools dropped a letter grade
2007 08 outlook
2007-08 Outlook
  • Adequate Progress of Low 25%
  • Writing Plus
  • Science and Writing (Use of District Averages in the Calculations)
  • School Grading Scale
  • Rating System for Alternative Schools
subgroups classifications
Subgroups Classifications

AYP calculations are based upon nine subgroups.

1. Total School

2. White (W)

3. Black (B)

4. Hispanic (H)

5. Asian (A)

6. Native American (I)

7. Economically Disadvantaged Students

8. English Language Learners (ELL)

9. Students With Disabilities (SWD)

39 components of ayp
39 Components of AYP
  • Participation Rate in Reading for the 9 subgroups
  • Participation Rate in Math for the 9 subgroups
  • Reading Proficiency of 9 subgroups
  • Math Proficiency of 9 subgroups
  • Change in School Writing Proficiency
  • Change in School Graduation Rate
  • School Grade
2006 07 state targets for ayp
2006-07 State Targets for AYP
  • 95% Participation Rate in Reading
  • 95% Participation Rate in Math
  • 51% of Students Proficient in Reading
  • 56% of Students Proficient in Math
  • 1% Improvement in Writing Proficiency or 90% at 3 and above
  • 1% Improvement in Graduation Rate or graduation rate of 85% or higher
  • School Grade not a D or F
determine if reading math proficiency targets were met
Determine if Reading/Math Proficiency Targets were Met
  • Three Ways to Compute:
    • First Method: Did the school and ALL subgroups meet the reading/math proficiency target?
      • If Yes: Proficiency Targets are met
      • If Not: Check Second Method
    • Second Method: Did the school and/or ALL subgroups that did not meet reading/math proficiency targets meet Safe Harbor requirements?
      • If Yes: Reading/Math proficiency targets are Met
      • If Not: Check Third Method
    • Third Method: Did the school and/or ALL subgroups that did not meet Safe Harbor requirements meet Growth Model “on track to be proficient” requirements?
      • If Yes: Reading/Math proficiency targets are Met
      • If Not: Reading/Math proficiency targets are Not Met
when can a school use safe harbor
When can a School Use Safe Harbor?
  • The Safe Harbor provision can be applied only if a school has met all four of the requirements below:
    • At least 95% of students in total and in each subgroup have participated in testing.
    • The school has met writing criteria.
    • The school has met graduation rate criteria.
    • The school grade is not D or F.
what are requirements for safe harbor
What are Requirements for Safe Harbor?
  • Safe Harbor requires that for each subgroup being evaluated:
    • The percent of non-proficient students decrease by at least 10% from the preceding year
    • Graduation Rate Criteria is met
    • Writing Criteria is met
when can a school use the growth model provision
When can a School Use the Growth Model provision?
  • The Growth Model provision can be applied only if a school has met all four of the requirements below:
    • At least 95% of students in total and in each subgroup have participated in testing.
    • The school has met the writing criteria.
    • The school has met the graduation rate criteria.
    • The school grade is not D or F.
what are requirements for the growth model
What are Requirements for the Growth Model?
  • The Growth Model requires that for each subgroup being evaluated:
    • The percent of students “on track to be proficient” in three years or less is at least 51% in reading and 56% in math and
    • The group has met the writing criterion (the increase in writing proficiency is at least 1% or the school has a writing proficiency rate of 90% or better) and
    • The group has met the graduation rate criterion (the increase in graduation rate is at least 1% or the school has a graduation rate of 85% or better).
impact of the growth model
Impact of the Growth Model
  • 675 schools met all proficiency benchmarks.
  • 114 schools met requirements using Safe Harbor.
  • 185 schools used the Growth Model or a combination of the Safe Harbor and the Growth Model.
  • 113 schools needed the additional help of the mathematical adjustment to meet AYP.
sv2 3 match process
SV2/3 Match Process
  • Two Critical Processes Performed by local MIS in Conjunction with EIAS
    • Matching Process
    • Data Update Process
  • EIAS matches records by district, school and SID/AID
  • Must be done through NWRDC – not a web process
purpose of matching
Purpose of Matching
  • For AYP and School Grading Process:
    • Identify students meeting the requirement of being in school all year.
      • In Survey 2 and Survey 3.
    • Identify students needing updates to data critical to the school grades and AYP calculations.
      • Race.
      • Grade Level.
      • SWD Code.
      • ELL Code and Entry Date.
      • FRL Code.
      • Withdrawals.
the five files on nwrdc
The Five Files on NWRDC
  • F70550Unmatched Problem Records
    • Students whose Prior School Status (PSS) shows same school, same district.
    • Students for whom the demographic record is missing a matching PSS record.
    • Students whose PSS withdrawal code is other than W01, W02, or ZZZ.
    • Students who did not match between Svy2 and Svy3.
    • Students with duplicated alias numbers.
  • F70551Unmatched Expected Records
    • Most likely these students are new to the state between surveys.
    • Withdrawal code of ZZZ and Entry/re-entry date after Svy2.
the five files on nwrdc cont
The Five Files on NWRDC (cont.)
  • F70549 Deleted Survey 3 Records and F70586 Deleted Survey 2 Records
    • McKay, Home Ed, & Private School students.
    • Be sure each student is identified correctly.
  • F70548 Matched Records
2007 08 timeline
2007-08 Timeline
  • Survey 3, 2007-08.
    • Survey Week: February 4-8, 2008.
    • State Processing: February 11 – March 7, 2008.
  • Correct errors and submit corrections by 4:00 p.m. EST on Friday, March 7, 2008.
need for improvement
Need for Improvement
  • Missing and Incorrect Data
    • Race, Gender, SWD, ELL, etc. not always updated.
    • ELL dates often missing.
    • Incorrect Grade Levels.
    • Students enrolled in a school where they do not receive the majority of their instruction.
to remember
To Remember
  • Submit all four records, as appropriate:
    • Student Demographic,
    • Exceptional Student (SWD)
    • English Language Learner (ELL)
    • Student Course Schedule
  • Students must have a course record to be included.
  • Data should reflect Friday of FTE Week (February 8).
tentative timeline for corrections processes
Tentative Timeline for Corrections Processes
  • School Types – January/February 2008
  • Prior Year Data – February  April 2008
  • Student Data Updates – March 13  April 2008
  • Retakes – March 13  April 2008
  • Assessment Corrections – June 2008
  • Appeals – Mid July to Mid August 2008
aids for gathering prior year data
Aids for Gathering Prior Year Data
  • Student Locator
    • Use CICS system at NWRDC
    • Locate student IDs and prior FL public schools attended
  • FASTER method of sending records electronically.
    • Keep Reading and Math Litho Codes locally
    • Send to new school when student moves
    • Required data elements as of 1/1/07
slide70

Prior Year Data Search Engine

  • Allows a search of FCAT assessments.
  • Search for Litho Codes.
  • Search by last year’s District, School, Last Name, First Name, Grade level.
  • An expanded Litho Code was necessary in 2007 to assure each record was unique.
    • 8 digit Litho Code +
    • 2 digit grade +
    • 1 digit Retake indicator—R=Retake; 0=Not a Retake.
    • Example: 12345678040 or 3456789810R.
overall improvements
Overall Improvements
  • The site will be made more secure.
  • Pages will be made more aesthetically pleasing.
  • Brief descriptions will be available (mouse over?) for clarification.
  • Links will be larger, logically arranged, and consistent among pages and applications.
  • Links for downloading views into Excel.
  • Links will be added where necessary to enhance navigation.
more improvements
More Improvements
  • Error messages will be more meaningful and consistent among applications.
  • A “View All Errors” page will be added to appropriate applications.
  • Page Titles and messages will be clear and appropriately placed.
  • Highlighting will be made more visible.
  • Instructions will be more user friendly.
  • Views will contain records unique to that view where appropriate.
page specific improvements
Page Specific Improvements
  • Login Page.
    • DOE Contact information will be put on this page.
  • Upload Page
    • A link to download the formats in Excel.
  • District and School Home Pages
    • School page will have a note to call district if a reset is needed.
  • Student Edit page
    • The back button may be disabled.
    • There will be a reset button that will reset all data entry cells to blank.
application specific improvements
Application Specific Improvements
  • School Types
    • Add a way of indicating new schools.
  • Prior Year Data Corrections
    • School dropdowns will be populated appropriately.
  • Student Data Updates
    • The “met graduation requirements” flag will be recoded.
    • Withdrawal flag will be recoded.
  • Assessment Data Corrections
    • A “view all records” page will be added.
  • Appeals
    • Data will be entered via the web rather than uploaded.
issues for 2007 going forward
Issues for 2007, Going Forward
  • Adequate Progress of Low 25%
  • Writing Plus
  • Science and Writing (Use of District Averages in the Calculations)
  • School Grading Scale
contact information
Contact Information

Evaluation and Reporting Office

Florida Department of Education

325 West Gaines Street, Room 445

Tallahassee, FL 32399

Website: http://www.fldoe.org/evaluation/

Email: EVALNRPT@fldoe.org