Early Childhood Tracking Systems Hands On Workshop ESO Network 14 (Jackie Young)

1. Early Childhood Tracking SystemsHands On Workshop ESO Network 14 (Jackie Young) Prepared by Deena Abu-Lughod, SAF November 29, 2007 Participants: 4, 7, 20, 24, 59, 81, 88, 197, 209, 230, 280

2. Agenda • Purpose • Rationale from SQR • Current Status • Data Availability (DIBELS, ECLAS 2, ORA, CBM) • Ideal System Criteria • Building a system using RESI • Evaluation

3. Purpose • Take Writing Prompt 1 • For 30 seconds, think about the prompt. • For 90 seconds, write down as much as you can about what you believe the purpose of today should be. Do not worry about spelling or punctuation or grammar. • Place your paper on the wall. • A volunteer will read all submissions. • What commonalities do you see? • As a group, formulate a statement that represents the consensus.

4. Rationale: Expectations from SQR • Quality Statement 1 – Gather Data: School leaders and faculty consistently gather and generate data and use it to understand what each student knows and is able to do, and monitor the student’s progress over time. To what extent do school leaders and faculty gather, generate, and utilize data􀂾 􀀳 􀂨 • 1.1 to provide an objective, constantly updated understanding of the performance and progress of each student, classroom and grade level in each subject area • 1.2 to provide an objective, constantly updated understanding of the performance and progress of special education students • 1.3 to provide an objective, constantly updated understanding of the performance and progress of English language learners • 1.4 to provide an objective, constantly updated understanding of the performance and progress of ethnic groups, boys and girls and all other categories of interest to the school • 1.5 to compare the school’s current and past performance and progress, and compare performance and progress of different students, classrooms and grades, and in different subject areas • 1.6 to compare the school’s performance and progress to that of similar schools • 1.7 with sufficient training, management systems and structures to enable data to be used effectively to inform planning and instruction and to track the progress of students over time

5. Rationale: Expectations from SQR • Quality Statement 5 – Monitor and Revise: The school has structures for monitoring and evaluating each student’s progress throughout the year and for flexibly adapting plans and practices to meet its goals for accelerating learning. • To what extent do . . . Δ 􀂾 􀀳 • 5.1 the school’s plans for improving student outcomes include interim goals that are objectively measurable and have suitable time frames for measuring success and adjusting • 5.2 the school’s plans for improving teacher outcomes include interim goals that are objectively measurable and have suitable time frames for measuring success and adjusting • 5.3 faculty use periodic assessments and other diagnostic tools to measure the effectiveness of plans and interventions for individual students and groups of students in key areas • 5.4 faculty use the information generated by periodic assessments and other progress measures and comparisons to revise plans immediately in order to reach stated goals • 5.5 school leaders track the outcomes of periodic assessments and other diagnostic measures and use the results to makes strategic decisions to modify practices to improve student outcomes • 5.6 school leaders and teachers use each plan’s interim and final outcomes to drive the next stage of goal setting and improvement planning • 5.7 the principal and school community have a clear vision for the future development of the school and implement procedures and systems to effect change

6. Current Status: Monitoring performance and progress • Read the following Indicators and think about how your school currently meets the expectations of the Quality Review: • 1.4: to provide an objective, constantly updated understanding of the performance and progress of ethnic groups, boys and girls and all other categories of interest to the school • 5.5: school leaders track the outcomes of periodic assessments and other diagnostic measures and use the results to make strategic decisions to modify practices to improve student outcomes

7. Current Status: Monitoring performance and progress Take Writing Prompt #2. • At the signal, think about the prompt for 30 seconds. • For 90 seconds, write as much as you can as a response. Do not worry about spelling, punctuation or grammar. • Share your ideas with your partner. (2 minutes) • Reverse roles. (2 minutes) • As a pair, identify 1 key idea for each indicator (or 2 ideas if you both wrote on the same indicator) and report out to the group.

8. Our Beliefs: How do children learn to write? • Take Writing Prompt #3 • For 30 seconds, think about the prompt. • For 90 seconds, write down as much as you can about how you believe children learn to write. Do not worry about spelling or punctuation or grammar.

9. Our Beliefs • Text Based Discussion based on “What’s Next for this Beginning Writer?”

10. Data Availability: DIBELS • Provides overall instructional recommendation and scores on sub-components (ISF, LNF, PSF, NWF, ORF, WUF) • Available as pdf and is exportable as a spreadsheet

11. Data Availability: ECLAS 2 • Online access (delayed) • Excel spreadsheet (through network) • High correlation between EOY Grade 3 Sight Words score and Grade 3 ELA • Indicates mastery levels for various subtests

12. Data Availability: ORA • Administered and scored by hand • No electronic tracking system developed

13. Data Availability: DRA/RIGBY • Administered and scored by hand • Tracking system under development

14. Data Availability: EDM Tracking • Developed by Teach for America • Each question on each unit test is coded by performance indicator and standard • Teacher enters data (1 for correct; 0 for incorrect) for each student and each item. • Spreadsheet automatically computes student and classroom average. • Format allows for easy detection of patterns by indicator • Requires some Excel skills

15. Data Availability: Other 1

16. Data Availability: Other 2

17. Constructing Criteria • Form groups based on Sets 1, 2 or 3. • What elements do you like about the tracking system? • What elements are troubling? • What changes would you make? • What would a complete system look like?

18. Elements of a good system

19. Create your own system • On the planning worksheet, identify the data elements you would like to include in your system. • Consider the order in which you would like the elements to appear. • When you are ready, load up the RESI report on your computer. Make a copy of it (so you can save the original in case you make a mistake).

20. Our own Writing Prompts

21. Evaluation • Review purpose • What did we achieve? • Do we need to continue? • Complete PMI (Plus, Minus, Interesting)

22. Progress Monitoring • Progress monitoring (PM) is conducted frequently and is designed to: • Estimate rates of student improvement • Identify students who are not demonstrating adequate progress • Compare the efficacy of different forms of instruction and design more effective, individualized instructional programs for problem learners

23. What Is the Difference Between Traditional Assessments and PM? • Traditional assessments: • Lengthy • Administered irregularly • Feedback is delayed • May lack meaningful contexts (eg. scores may be based on national norms and a teacher’s classroom may differ from the national student sample)

24. What Is the Difference Between Traditional Assessments and CBM? • Curriculum-Based Measurement (CBM) is one way to monitor progress: • CBM provides an easy and quick method to measure student progress • Teachers can analyze student scores and adjust student goals and instructional programs • Student data can be compared to by classroom or subgroup

25. Curriculum-Based Assessment • Features: • Materials align with school curriculum • Measurement is frequent • Assessment information can be used to formulate instructional decisions

26. Basics of CBM • Monitors student progress across entire school year • Meaningful, accurate results can help quantify short- and long-term student gains • Probes for reading, spelling, writing and math are brief and easy to administer • Teachers quickly determine whether an intervention is helping a student

27. Steps for Conducting CBM • Step 1: Place Students in a Task for Progress Monitoring • Step 2: Identify the Level for Material for Monitoring Progress • Step 3: Administer and Score Probes • Step 4: Graph Scores • Step 5: Set Goals • Step 6: Apply Decision Rules toGraphed Scores to Know When to Revise Programs and Increase Goals • Step 7: Use the Data Qualitatively to Describe Students’ Strengths and Weaknesses

28. Step 1: Place Students in a Written Expression Task for Progress Monitoring • Decide which task or grade level of material is developmentally appropriate • Monitor writing fluency by using Written Expression CBM in Grades 1–12 (as soon as students can write sentences)

29. Step 2: Identify the Level of Material for Monitoring Progress • For Written Expression: • Prompts should be simple in sentence structure and tap age-appropriate background knowledge of students • Prompts should represent experiences of a wide range of school-age students

30. Step 3: Administer and Score Written Expression CBM • Administer to entire class at one time • Present students with a story starter: • Use a familiar theme • End the starter mid-sentence • Write the starter at top of student CBM probe • Give students a set time to write • Score Written Expression CBM probes

31. Sample Probes • I was on my way home from school and … • I was talking to my friends when all of a sudden … • It was a dark and stormy night … • One day I found the most interesting thing … • One night I had a strange dream about … • I found a note under my pillow that said … • The cave was very dark and … • One day I went to school but nobody was there except me …

32. Step 3: Administer and Score Written Expression CBM • Read the story starter aloud to the class • Give students 30 seconds to think about their writing • Clarify whether students can draw or write notes • Re-read story starter aloud to the class • Students write for 3 minutes (early childhood)

33. Step 3: Scoring Written Expression • Several ways to score: • Total words written • Total words spelled correctly • Total words written minus incorrectly spelled words • Correct word sequences (CWS) • Correct minus Incorrect Word Sequences (CIWS) • Keep the scoring system consistent throughout the entire school year

34. Step 3: Administer and Score Written Expression CBM • Words written: • Calculate the total number of words written • Correct spelling, word usage, capitalization, and punctuation are ignored

35. Step 3: Administer and Score Written Expression CBM The cave was very dark and I try to close my eyes, so I couldn’t see anything, but that didn’t help. Than I hear some one breathing. I try to stream, but nother came out. The breathing became close and close to me, and the worst Part was that I couldn’t see athing. At first I thought meslef that I an Just emaging stuff.

36. Step 3: Administer and Score Written Expression CBM I was on my way home from school and I saw my friend and we rund bilk to my hous. Bot she had to go to the bath room at the palk. So we played a little bit and we whant to the store and bot some goodes.

37. Step 3: How to Administer and Score Written Expression CBM I was on my way home from school and I saw my friend and we rund bilk to my hous. Bot she had to go to the bath room at the palk. So we played a little bit and we whant to the store and bot some goodes.

38. Step 3: Administer and Score Written Expression CBM • Words spelled correctly: • Calculate the total number of words spelled correctly • Any correctly spelled English word is counted as correct • Proper usage, capitalization, and punctuation are ignored

39. Step 3: How to Administer and Score Written Expression CBM The cave was very dark and I try to close my eyes, so I couldn’t see anything, but that didn’t help. Than I hear some one breathing. I try to stream, but nother came out. The breathing became close and close to me, and the worst Part was that I couldn’t see athing. At first I thought meslef that I an Just emaging stuff.

40. Step 4: Graphing Scores • Once CBM data has been collected, graph student scores • Graphs provide teachers with a straightforward way of: • Reviewing student progress • Monitoring appropriateness of student goals • Judging adequacy of student progress • Comparing and contrasting successful and unsuccessful instructional aspects

41. Step 4: Graphing Scores • Teachers can make decisions about short- and long-term progress • Graphs help teachers set ambitious, but realistic goals • Graphs provide teachers with actual data to help revise and improve student instructional programs

42. Step 4: Graphing Scores • Two options for creating CBM graphs: • Create own students’ graphs using graph paper and pencil • Create graphs using computer software (such as Microsoft Excel) • Create a master graph • Vertical axis: range of scores of all students in the class • Horizontal axis: number of weeks of instruction • Master graph serves as a template Create a graph for each individual student for interpretation

43. Step 4: Graphing Scores

44. Step 5: Setting Goals • End-of-year performance goals • Three options: • End-of-year benchmarking • Intra-individual framework • Norms

45. Step 5: Option #1 • Option #1: End-of-year benchmarking • Identify end-of-year benchmark • Mark as an “X” on the graph at the date marking the end of the year • Goal-line is drawn between median of first three CBM scores and end-of-year performance goal

46. Step 7: Use the CBM Database Qualitatively to Describe Student Strengths and Weaknesses • Written Expression CBM

47. Step 7: Use the CBM Database Qualitatively to Describe Student Strengths and Weaknesses • Written Expression CBM

48. Step 7: Use the CBM Database Qualitatively to Describe Student Strengths and Weaknesses • Written Expression CBM