Assessment Update

Assessment Update Prepared for the Florida Organization of Instructional Leaders (FOIL) May 15, 2008 Dr. Cornelia S. Orr Office of Assessment Florida Department of Education

Overview • FTCE/FELE Program Updates • FCAT Writing+ 2008 Results Summary • FCAT Writing+ Changes • Legislation and Assessment • FCAT Revision Cycle • Other Topics • Computer-Based Testing • MAP Clearinghouse

FTCE/FELE Fees • Effective January 1, 2009.

FTCE Middle Grades Integrated Curriculum (MGIC) Examination • Issue: • MGIC certification (grades 5-9) allows teachers to teach Algebra Honors, Geometry Honors and Physical Science Honors high school credit courses. • Teachers can earn this certification by passing the MGIC exam, but can miss all items of one core subject area—yet still be considered certified to teach all subject areas. • Actions: • Short term: Districts may require subject area certification (e.g., MG Math) • Long term: MGIC Competencies and Skills are currently being revised and this issue will be addressed.

FCAT Writing+ 2008 Results Summary (Percent >3.5) In 2008, 77% of Grade 4 students, 88% of Grade 8 students, and 78% of Grade 10 students scored at 3.5 and above on the essay portion of FCAT Writing+.

FCAT Writing+ 2008 Results Summary (Percent >4)

Some District Concerns • Grade 4 Modes: Expository – Narrative • Large differences between prompts (Narrative lower) • Large year-to-year differences for some schools • Examining the concerns • Review the essay scoring criteria and their use in 2008 • Examine the available statistics • Average scores, • IRR, Validity, assignment of scorers, number of third reads, and unscorable papers, • Percent >3.5 and 4.0, and • Distributions of scores • Review specific concerns (districts, schools, and students) • Expedite the delivery of the Writing+ CDs

Writing+ Grade 4 Scoring Observations • Scoring criteria for 2008 are linked to scoring criteria used in 1993. • The 2008 narrative prompt directed students to write a story about what happens when someone plays a game with family or friends. • Some students assigned to this prompt wrote expository responses that listed the process for playing a game, or they wrote about various things that happen while playing a game. • However, exposition or a description of a game were accepted as “on topic” if they provided information about the event or events. • Students were credited with writing "stories" because their presentation of how to play the game followed a time sequence. Although a clear story line may not have been evident, the time sequence provided enough narrative structure for the response to be scored in the higher ranges of the score points. • The 2008 anchor sets for the two modes are very comparable in what they expect a student to do at each of the score point levels. http://fcat.fldoe.org/fwprom08.asp • Students seemed to respond more passionately to the narrative prompt, while the expository prompt generated more list-like and less well developed responses. • Students writing narrative used a less "prescribed" writing style. That is, they seemed to own their writing and use a variety of original approaches. • Students writing expository used a variety of "rehearsed" or "coached" strategies that often limited scores in the higher ranges. • Because narrative writing can be used to explain, mixed mode responses in expository were often eligible for a score of 4, but students were usually unable to manage the response well enough to earn a score of 5 or 6.

Writing+ Average Score Trends

Examine Available Statistics

Writing+ Grade 4 – Score Distributions

Writing+ Summary • We need more information about school and student circumstances. • Grade 4 scorers were highly consistent with each other and with the standards established by Florida educators. • The assignment of raters was well distributed across districts and schools. • The summary statistics on the prompts are similar across prompts and approximate the field test stats. • The score “percent >3.5” is more sensitive to changes in scores in the middle of the distribution. • Expository – more 4’s (3.5 received one 4) • Narrative – more 5’s and 6’s • Instructional Implications - ?

FCAT Writing+ Changes • SBE Concurrence (4/15/08): Remove the MC items from the FCAT Writing+ tests and delay its use as a graduation requirement. • Rationale: • Budget Reduction – Multiple-choice questions do not impact our state or federal accountability systems. • Technical concerns – Inconsistencies in 2007 between scores on the essay and MC questions, making the combination into a single score problematic for high stakes student decisions (10th). • SB 1908: Removes MC items beginning 08-09 • Next Steps: • Revise SBE Rule 6A-1.09422 to delay FCAT Writing+ graduation requirement • Revise pupil progression plans

Legislation and Assessment • FCAT Test Preparation • NRT and impact on Progression Plans • SB 1908 removes the requirement that the statewide assessment program include norm-referenced tests. • End-of-Course exams • SB 1908 allows the Commissioner to incorporate end-of-course assessments into the statewide assessment program, in addition to the comprehensive assessments of reading, mathematics, writing, and science.

Legislation and Assessment • FCAT Writing+ • Beginning with the 2008-2009 school year through 2011-2012, SB 1908 requires the Commissioner to discontinue the administration of the multiple-choice items on the comprehensive assessment of writing. In 2012-2013, the Commissioner must administer a comprehensive assessment of writing with specific characteristics, including a combination of MC, SR, and ER items

Legislation and Assessment • CPT Exam for 11th graders • SB 1908 requires high schools to evaluate the college readiness of selected students prior to grade 12, beginning with the 2008-2009 school year, based on minimum test scores determined by the SBE. • Students achieving the minimum scores, and enrolling in a community college within 2 years, will not be required to enroll in remediation courses. • High schools, to the extent practicable, must provide 12th grade students scoring below the minimum scores with access to remedial instruction prior to graduation.

FCAT Revision Cycle

Other Topics • Use of Computer-Based Testing • MAP Clearinghouse • Questions?

Assessment Update