Administration and Scoring of Spelling Curriculum-Based Measurement CBM for use in General Outcome Measurement

2. Administration and Scoring of SpellingCurriculum-Based Measurement (CBM)for use in General Outcome Measurement

3. Part of a training series developed to accompany the AIMSweb Improvement System. Purpose is to provide the background information and data collection procedures necessary to administer and score Spelling Curriculum Based Measurement (S-CBM). Designed to accompany: AIMSWeb Training Workbook: Administration and Scoring of Spelling CBM for Use in General Outcome Measurement Standard Spelling Probes AIMSweb Web-based Software Training Video Overview of Spelling-CBM Assessment Training Session

4. Brief review of Curriculum Based Measurement (CBM) and General Outcome Measurement (GOM). Its Purpose Its Origins Learn how to administer and score Spelling - Curriculum Based Measurement (S-CBM) through applied practice. Training Session Goals

5. General Outcome Measurements from Other Fields Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6

6. Common Characteristics of General Outcome Measures

7. CBM is Designed to Provide Educators With… The same kind of evaluation technology as other professions! Powerful measures that are: Simple Accurate Efficient indicators of student achievement that guide/inform a variety of decisions Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6 Administration and Scoring of Spelling Curriculum– Based Measurement for Use in General Outcome Measurement Training Workbook p. 6

8. Not a measure of performance on a particular lesson or unit A broader indicator Level of general spelling skills relative to peers Degree of progress in spelling relative to self or peers General Outcome Measures in Spelling

9. “Signs” of general achievement Not everything Important things Standardized tests Administration Scoring Interpretation Researched Reliability and validity Accuracy of measurement Sensitive to improvement in short periods of time Designed to be as short as possible (do-ability) Linked to decision making for promoting positive achievement Things to Remember About General Outcome Measures in Spelling

10. Traditional spelling tests versus S-CBM Shorter duration (2 minutes) Spelling words dictated carefully at set pace Random sample from a large pool of graded spelling words Combination of words Words students have already learned Words they will learn to spell in upcoming weeks or months Exact and sensitive way to assess spelling improvement Correct letter sequences Correct word spellings Spelling CBM

12. Timing 2 minutes New word dictated every 7 seconds (every 10 seconds: grades 1 & 2) Arrangements Group Individual What is scored Number of letters spelled in correct sequence Number of words spelled correctly Summary of Spelling Conditions

13. Students have lined paper numbered 1–12 (Grades 1 & 2) or 1–17 (Grade 3-8)and a pencil. Say, “We’re going to take a 2-minute spelling test. I am going to say some words that I want you to spell on the sheet of paper in front of you. Write the first word on the first line, the second word on the second line and so on. I’ll give you [10] [7] seconds to spell each word. When I say the next word, write it down, even if you haven’t finished the last one. You will receive credit for each correct letter written. Are there any questions? [Pause] Let’s begin.” Say the first word and start your stopwatch (or tape recorder) Standard Directions

14. Say each word twice. Use homonyms in a sentence (provided) Say a new word every 10 or 7 seconds Say, “You should be on the fifth word which is…” Monitor students to ensure they are writing on the correct line Do not respond to student questions during the test After two minutes, say, “Stop. Put your pencils down.” Good to begin with short practice test to reduce novelty. Shortened directions available for well-experienced students. Standard Directions

15. Benchmark: 3 standards lists for each grade (1–8) Screen and identify students who are at risk Monitor progress and improvement in fall, winter, and spring Guide program evaluation decisions & improve accountability Progress Monitor: 30 standard lists for each grade Provide practical way to write individualized progress goals Monitor progress and determine rate of improvements Provide accountability data Standard Assessment Spelling Lists

16. An Example

18. Student needs: Numbered, lined paper with the exact number of lines (12 or 17) Saves time Reduces potential confusion Makes scoring easier Answer sheets available as PDF documents Pencil

19. Examiner needs List of words with time interval clearly marked Includes sentences for homonyms Improves ease of scoring Timing device

20. Download and review S-CBM Manual Download and reproduce student answer sheets Download Standard Spelling Assessment Lists Curriculum independent Allow decision making Minimize between-teacher, between-school differences Stopwatch or timer Clock on the wall is inaccurate, inefficient Timing tape Audio tape with recorded tones at prescribed intervals Allows better attention to words and students List of students to assess

21. Quiet, without distractions May be 1:1, small group, whole classroom Consider a simple practice test Use very simple words Allows students to adjust to pace Show student how they get credit for unfinished words Be sure you can monitor Especially those students who “get lost” easily Be available to point to the number of the next word

22. Keep examiner out of the process Keep number of words said to minimum Some students try to spell everything said Only homonyms have sentences (say word, say sentence, repeat word) Roam and project Loud enough for all students to hear Be sure head is turned toward students Monitor, monitor, monitor Alert for lost students Before they become frustrated and quit Before they call out that they are lost Available to point to the next number

23. Say the number of the word periodically Younger students, say, “You should be on the fifth (tenth) word which is…” Older students find this intrusive, so don’t say the numbers Avoid answering questions during testing Students may comment: You’re going too fast, I’m lost, What number are we on? Deal with this in practice tests Point to orient student Adhere to end of timing Interruptions Re-administer later with another spelling list Consider that some students need small group or 1:1

24. Helpful Hints Ensure a quiet and distraction-free testing environment Students must be able to hear the examiner Offer no assistance, even when you know the child is capable of more Complete reliability checks Before and after data collection with all examiners to ensure reliable administration and scoring Additional reliability checks at least once a year Prevent examiner “drift” Use same Benchmark examiner for the same students. Younger children tend to perform better with familiar examiners. • Make sure the testing environment is quiet and free from distractions. Students must be able to hear the examiner during the Phonemic Segmentation Fluency task. • Complete reliability checks before data collection with all examiners to ensure reliable administration. Additional reliability checks should be done at least once/year for all examiners, even those familiar with testing. • If possible, try to have the same examiner during each Benchmark period test the same students. Younger children tend to perform better with familiar examiners. Periodic reliability checks will help with examiner “drift,” or a tendency to provide additional assistance when you know a child is capable of completing more of each task.• Make sure the testing environment is quiet and free from distractions. Students must be able to hear the examiner during the Phonemic Segmentation Fluency task. • Complete reliability checks before data collection with all examiners to ensure reliable administration. Additional reliability checks should be done at least once/year for all examiners, even those familiar with testing. • If possible, try to have the same examiner during each Benchmark period test the same students. Younger children tend to perform better with familiar examiners. Periodic reliability checks will help with examiner “drift,” or a tendency to provide additional assistance when you know a child is capable of completing more of each task.

25. Follow the standardized directions: Spelling-CBM is a standardized test Administer the assessment with consistency Remember it’s about testing, not teaching Don’t teach or correct Never practice using the standard spelling lists Things You Need DURING Testing

26. Things to Do AFTER Testing Score Immediately! Determine the words spelled correctly Circle the number beside the words spelled correctly Award full points Determine the correct letter sequences for misspelled words(see next slide) More likely to show change Provides diagnostic and positive feedback

27. Spelling Scoring Correct: Count each sequence of 2 letters Spaces before and after the word count as “letters” Consider: break through versus breakthrough Example: cat c a t

28. Spelling Scoring store _ s t o r e _ (6) stor _ s t o r e _ (6) fishing _ f i s h i n g _ (8) fisshing _ f i s s h i n g _ (8) cat _ c a t _ (4) catz _ c a t z _ (4)

29. Spelling Scoring store _^s^t^o^r^e^_ 6 (6) stor _^s^t^o^r e _ 4 (6) fishing _^f^i^s^h^i^n^g^_ 8 (8) fisshing _^f^i^s s h^i^n^g^_ 7 (8) cat _^c^a^t^_ 4 (4) catz _^c^a^t z _ 3 (4)

30. Spelling Scoring, Continued Tricks: Use master sheet with all correct sequences pre-counted. Then you only need to count the scoring for the few incorrect sequences (errors) on each paper. A complete list of spelling rules can be found in the Appendix of your workbook. Please review and become more familiar with the more unusual errors.

31. Summary You now have the building blocks to begin Spelling - CBM to ensure literacy growth. Practice to Automaticity — You’ll get more efficient Get Checked Out by a colleague using the Accuracy of Implementation Rating Scale (AIRS) for accuracy and efficiency Stay in Tune by periodically checking AIRS

32. Let’s Practice

33. Practice: Spelling Open your workbooks to Spelling Practice List 1: Elida Score this spelling test Words spelled correctly Correct letter sequences

34. Answer:ElidaWSC = 13CLS = 121

35. If the Student Misses a Few Words… Any missing single letter equals 2 missing CLS. Any 2 letters together that are missing equals 3 missing CLS Any incorrectly inserted letter equals 1 missing CLS

36. Answer:LesterWSC = 0CLS = 52

37. Answer:CalvinWSC = 9CLS = 109

38. Determining Inter-Scorer Agreement Agreements / (Agreements + Disagreements) x 100 Example: Examiner 1 = 90 CLS Examiner 2 = 92 CLS Inter-Scorer Agreement = 98%

39. Answer:LucindaWSC = 2CLS = 70