- By
**thom** - Follow User

- 91 Views
- Uploaded on

Screening for Mathematics Difficulty. Ben Clarke, Ph.D. November 6, 2007.

Download Presentation
## PowerPoint Slideshow about 'Screening for Mathematics Difficulty' - thom

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Using Curriculum-Based Measurement (CBM) in Elementary Mathematics

### Introduction and Overview of Progress Monitoring

### Common Assessment Approaches

### Examples of Progress Monitoring Measures

### Examples of Robust Indicators for

The Center on Instruction is operated by RMC Research Corporation in partnership with the Florida Center for Reading Research at Florida StateUniversity; Horizon Research, Inc.; RG Research Group; the Texas Institute for Measurement,Evaluation, and Statistics at the University of Houston; and the VaughnGross Center for Reading and Language Arts at the University of Texas at Austin.The contents of this PowerPoint were developed under cooperative agreement S283B050034 withthe U.S. Department of Education. However, these contents do not necessarilyrepresent the policy of the Department of Education, and you should notassume endorsement by the Federal Government.2007 The Center on Instruction requests that no changes be made to the content or appearance of this product.To download a copy of this document, visit www.centeroninstruction.org

A lesson from reading: Perhaps?

- The trajectory of reading development led to a focus on prevention efforts
- Preventing the establishment of reading problems is now accepted practice dependent on…
- Early identification assessments
- Effective early literacy interventions
- Does the same pattern hold true for mathematics?

What about math?

- Does math develop in the same way as reading? That is, does it make “sense” to prevent math problems?
- Less knowledge about long term trajectories
- Emerging databases may help to answer this question (ECCLES, Jordan, Fuchs)
- Emerging evidence suggests difficulty in math is relatively stable over time

What about math?

- We do know that math knowledge develops across multiple domains (e.g. number, geometry)
- Students may be on-track in one area and at-risk in another
- How these domains develop and interact continues to be researched
- The importance of a logical scope and sequence is paramount

Characteristics of students with mathematics difficulties

- Different profile types exist for students with mathematics difficulties
- Students with mathematics difficulties often have
- difficulty with quick and accurate retrieval of basic facts (Hasselbring, 88)
- Working memory deficits (Geary, 94)
- Difficulty in abstracting mathematical meaning from symbols

Early Screening

- Brief screening measures
- Used with all students to screen for risk status
- Use with all students drives the design and construction of screening measures
- Measures are of short duration and may be timed

Early Screening Cont.

- Goal is to maximize the amount of information collected in the minimum amount of time
- Predictive validity is critical (e.g. a low score in the Fall predicts difficulty at the end of the year)
- Predictive validity is to a broad measure of mathematics. That is we are predicting general outcomes.
- Screening instruments have been developed by a number of researchers and focus primarily on critical aspects of numerical proficiency and number sense

Number Sense

- Critical early mathematical skills may be centered around the concept of number sense.
- Number sense has been defined as:

“a child’s fluidity and flexibility with numbers, the sense of what numbers mean, and an ability to perform mental mathematics and to look at the world and make comparisons”

(Gersten & Chard, 1999)

Number Sense (Case, 98)

- Fluent, accurate estimation and judgment of magnitude comparisons.
- Flexibility when mentally computing.
- Ability to recognize unreasonable results.
- Ability to move among different representations and to use the most appropriate representation.

Numerical proficiency skills for early screening

- Strategic counting
- Magnitude comparison
- Number combinations
- Word problems
- Number identification (gateway skill)
- Sequence Counting (gateway skill)

Examples of Screening Measures

- Research line by Clarke and colleagues
- Four measures for K and 1st grade
- EN-CBM Oral Counting measure
- Students orally count for one minute. No student materials.
- EN-CBM Number Identification measure

4 1

5 11

9 4

__ 13 14 6 __ 8 3 4 __

Early Numeracy CBM- EN-CBM Quantity Discrimination measure (Magnitude Comparison)
- EN-CBM Missing Number measure (strategic counting)

Results

- Number Identification
- r=.62-65 (K) and .40-.72 (1st)
- Quantity Discrimination
- r=.68-73 (K) and .43-.58 (1st)
- Missing Number
- r=.61-63 (K) and .43-.72 (1st)

Bryant

- Concurrent validity correlations
- Quantity Discrimination
- 0-99 first grade
- 0-999 second grade
- Included equal concept
- Individually administered (1st) group (2nd)
- r=.63 (1st) and .49 (2nd)
- Missing Number
- r=.58 (1st) and .60 (2nd)

Bryant Results

- Place Value
- r=.64 (1st) and .63 (2nd)
- Individually administered
- Addition/Subtraction combinations
- r=.55 (1st) and .59 (2nd)
- Group administered

Examples with Key Variables

- Simple word problems (Jordan, 05)
- Jill has two pennies. Jim gives her one more penny. How many pennies does Jill have now?
- Mark has three cookies. Colleen takes away one of his cookies. How many cookies does Mark have now?
- Magnitude comparison/Number Identification (Mazzocco, 05)
- Four items that predicted 3rd grade performance (also included number constancy and mental addition )

Single Skill to Multiple Skill

- Focus on a broader array of skills
- Often specific to one domain (e.g. number) OR
- Tied to curriculum objectives for a specific grade level

Number Knowledge Test

- Developed by Case, Okamato, and Griffin
- Contains four levels
- Students advance to the next level if they score above a criterion at the previous level
- Each level introduces new problems but also repeats problems with larger numbers

Number Knowledge Test: Cont.

- Level 0
- Here are some circles and triangles. Count just the triangles and tell me how many there are.
- Level 1
- If you had 4 chocolates and someone gave you 3 more, how many chocolates would you have?
- Which is bigger: 5 or 4?

Number Knowledge Test: Cont.

- Level 2
- Which is bigger: 19 or 21?
- What number comes 4 numbers before 17?
- Level 3
- What number comes 9 numbers after 999?
- Which difference is smaller: the difference between 48 and 36 or the difference between 84 and 73?

Curriculum Based Measurement (CBM)

- Fuchs and Colleagues
- CBM - Computation
- CBM - Concepts/Applications
- Also experimented with Number Combinations (addition/subtraction 0-12) and Number Counting/Identification (4, 5, 6, _, _)

Results

- Screening in Fall of first grade
- Conducted logistic regression analyses predicting end of second grade math disability status
- Broad based concepts&applicatoins measure was best predictor
- Caution: Low correlations overall (r=.40/.44)

Progress Monitoring Measures

- Progress monitoring measures
- Some early screening measures have features that also enable them to be used in progress monitoring (e.g. Fuchs - CBM computation)
- Other screening measures do not have features to enable progress monitoring (e.g. Mazzocco)
- Other measures are still being investigated as potential progress monitoring measures (e.g. Bryant)

Considerations

- Timed measures vs. Untimed measures
- Skill specific measures vs. Multiple skill measures
- Domain specific or across domains
- Long term-predictive validity

Future Research

- Examine and develop longitudinal databases
- Construct analysis of early mathematics measures (what are we really measuring)
- Investigate sensitivity of measures to model growth

Dr. Erica Lembke

University of Missouri

Dr. Pamela M. Stecker

Clemson University

Overview of the Presentation

- Introduction and overview of progress monitoring
- What it is and what it isn’t
- Advantages over other types of assessment
- Specific features and uses of mathematics progress monitoring
- Procedures for implementation
- Research findings
- Measures and sources
- Implications for practice

A word of thanks to contributors…

- COI progress monitoring in mathematics powerpoint slides: Anne Foegen, Pamela Stecker, & Leanne Ketterlin-Geller. Access at: centeroninstruction.org, click on math
- National Center on Student Progress Monitoring: www.studentprogress.org

Adapted from NCSPM

Progress Monitoring

- Supports formative evaluation of student learning
- Informs teacher instructional decision making

General Definition of Student Progress Monitoring

- Collecting and evaluating data to make decisions about the adequacy of student progress toward a goal
- Evaluating student rate of change (slope) as compared to the slope of anticipated progress
- Informing teacher planning for instruction

General Definition of Student Progress Monitoring

- Requires:
- Technically sound measures
- Multiple forms of the same measure
- Assessment systems that are sensitive to student growth
- Standardized administration procedures
- Frequent measurement (occurs at least monthly)

Display of ProgressMonitoring Data

Adapted from NCSPM

That ARE NOT Progress Monitoring

Common Assessment Approaches That Are Not Progress Monitoring

- Screening tools
- Diagnostic assessments
- Curriculum-embedded assessments
- Teacher created
- Publisher developed

Curriculum-Embedded Assessments

- Help teachers identify whether students learned a particular concept/skill or what was taught in the chapter or unit
- Track mastery of short-term instructional objectives
- Sampling of items is representative of a limited set of problems, concepts, or skills
- Assessment materials mirror instructional materials

Teacher Use of Curriculum-Embedded Assessments

- Teacher-created
- Teacher develops assessments that focus on a particular concept or skill
- Teacher creates multiple forms
- Teacher gives assessment until student has learned that skill or concept
- Teacher often uses with students who are struggling with particular concepts or skills

Teacher Use of Curriculum-Embedded Assessments

- Publisher-developed
- Teacher gives chapter and unit exams included with the textbook series to evaluate student learning
- Typically used with the entire class

An Example from an Elementary Tutoring Context

- Mr. Jones is tutoring a fourth grade student who struggles with math computation skills
- He examines the sequence of skills for fourth grade computation and develops a criterion-referenced test for each skill within the sequence

An Example from an Elementary Tutoring Context

- Mr. Jones provides instruction and gives alternate forms of the criterion-referenced test until the skill is learned
- Then he changes instruction to focus on the next skill in the sequence

Hypothetical Fourth-Grade Computation Curriculum

- Multidigit addition with regrouping
- Multidigit subtraction with regrouping
- Multiplication facts, factors to 9
- Multiply 2-digit numbers by a 1-digit number
- Multiply 2-digit numbers by a 2-digit number
- Division facts, divisors to 9
- Divide 2-digit numbers by a 1-digit number
- Divide 3-digit numbers by a 1-digit number
- Add/subtract simple fractions, like denominators
- Add/subtract whole number and mixed number

Hypothetical Fourth-Grade Computation Curriculum

- Multidigit addition with regrouping
- Multidigit subtraction with regrouping
- Multiplication facts, factors to 9
- Multiply 2-digit numbers by a 1-digit number
- Multiply 2-digit numbers by a 2-digit number
- Division facts, divisors to 9
- Divide 2-digit numbers by a 1-digit number
- Divide 3-digit numbers by a 1-digit number
- Add/subtract simple fractions, like denominators
- Add/subtract whole number and mixed number

Multiplication

Facts

Multidigit

Addition

10

8

Number of problems correct

in 5 minutes

6

4

2

0

4

8

10

12

14

6

2

WEEKS

Mastery of Multidigit Addition

and Subtraction

Potential Difficulties withCurriculum-Embedded Assessment

- Assessments do not reflect maintenance or generalization of the concepts/skills.
- Assessments typically are designed by teachers or sold with textbooks with unknown reliability and validity.
- Number of concepts/skills or chapters passed does not relate well to performance on high-stakes tests.

Potential Difficulties withCurriculum-Embedded Assessment

- Sequence of concepts/skills or chapters is logical, not empirical.
- Difficulty of tasks may vary from test to test.
- Performance on limited-skill assessments can be misleading.

Progress Monitoring

- The process of collecting and evaluating data to determine whether students are making progress toward instructional goals and/or are responding to instructional interventions

Features of Progress Monitoring Systems

- Data are collected and evaluated frequently
- Schedule is determined by current level of student performance and goal
- Frequency of assessment typically ranges from two times per week to monthly

Features of Progress Monitoring Systems

- Teachers may choose to monitor the progress of all students in class
- Typically, students who are at risk of failure are assessed until they reach proficiency
- Data-based decision rules are applied to graphed data to determine when goals should be raised or instruction should be modified

Features of Progress Monitoring Measures

- Difficulty of tasks remains consistent across the year
- Allotted time typically does not allow for completion of test, so student growth still can be assessed

Features of Progress Monitoring Measures

- Uses standardized administration and scoring
- Test administration is timed (relatively short tests in duration)
- Specific scoring rules are applied
- Scoring typically uses counts, rather than percent correct

Progress Monitoring

- Uses:
- Estimate rate of student improvement
- Describe student response to instructional methods
- Inform teachers about instructional decision making
- Aid teachers in targeting areas/skills that need remediation
- Help teachers build potentially more effective instruction for particular students

Steps for Data-Based DecisionMaking Using Progress Monitoring

- Decide on level of implementation (individual, small group, classroom, grade-level, school-level, or district-level)
- Decide on which measures to use
- Collect screening or baseline data
- Decide on short-term objective or end criteria
- Set long range goal
- Decide how often to monitor
- Administer timed, alternate measures
- Graph data
- Make instructional changes using decision-making rules
- Continue monitoring

This graph shows that three instructional modifications have been implemented to the original instructional program for Donald.

Using progress monitoring data to test effectiveness of adaptations to class instruction

Over Two Decades of Programs of Research in Mathematics

- Identification of reliable and valid measures for screening and progress monitoring
- Monitoring at a variety of grade levels
- Monitoring students with special needs
- Use of data-management software
- Application of data-based decision rules
- Teachers’ use of skills analysis (particularly with Monitoring Basic Skills Progress program)
- Use of instructional recommendations
- Pairing with instructional interventions, such as Peer-Assisted Learning Strategies, in general education classrooms

Summary of ResearchFindings

- Technically adequate measures are available for screening and progress monitoring
- More tools currently available at the elementary level, although recent work has addressed both early numeracy and algebra at the high school level
- Both students with and without disabilities have been included in research, and progress monitoring appears to contribute to student achievement when
- Teachers use data-based decision rules
- Teachers use the data to inform instructional decision making

Summary of ResearchFindings

- Teachers have expressed concerns about time constraints
- Teachers tend to be more satisfied and efficient when computer or Web-based systems are available for data management and/or data collection
- Analysis of student performance on the specific skills included in the repeated measures may aid teachers in developing instructional interventions
- Teachers appear to benefit from instructional recommendations being included as a part of the ongoing progress monitoring system or as a part of consultation with other professionals (e.g., address more skills, teach more one to one or small groups, use more motivational systems)

Summary of ResearchFindings

- Use of progress monitoring in conjunction with implementation of Peer-Assisted Learning Strategies (a form of classwide peer tutoring) contributes to enhanced achievement for students with different achievement histories, although incorporating strategies of elaborated feedback (e.g., paying attention to the partner’s work, explanation of how the answer was attained, asking for help) and conceptual explanations (e.g., using manipulatives to demonstrate problems, building number sentences to incorporate real-life scenarios, discussing why problems need to be worked a certain way) seemed to contribute to student achievement even more.

Research Supports the Use of Progress Monitoring

- Progress monitoring data produce accurate, meaningful information about student academic levels and corresponding rates of improvement
- Progress monitoring data are sensitive to student improvement

Research Supports the Use of Progress Monitoring

- Performance on progress monitoring measures corresponds well to performance on high-stakes tests
- When teachers use progress monitoring data to inform their instructional decisions, students make greater learning gains

Two Approaches to Developing Progress Monitoring Measures(Fuchs, 2004)

- Curriculum Sampling
- Systematically sample items from the annual curriculum on each measure
- Robust Indicator
- Identify a global behavior that either encompasses many skills taught in the annual curriculum or is predictive of proficiency in the annual curriculum

Curriculum Sampling

- Each probe is a proportional sampling of the annual curriculum
- Advantages
- May conduct skills analysis
- May evaluate maintenance and generalization of skills
- Disadvantages
- Tend to be longer in duration
- May not generalize to other curricular programs
- Are grade-level specific

Robust Indicators

- Also referenced as general outcome measures
- Probes are comprised of tasks that represent proficiency in the content domain
- INDICATORS; not the “whole” of instruction
- Examples: oral reading fluency; estimation
- Empirically determined through correlations with other indicators of proficiency in mathematics

Robust Indicators

- Advantages
- Do not have to be grade specific
- Tend to be shorter in duration
- May be used across curricular programs
- Disadvantages
- May not be tied closely to instructional content
- May not be able to provide skills analysis on instructional content
- May not be able to evaluate maintenance and generalization of instructional skills

Measuring Mathematics Progress of Elementary Students

- Mathematics measures for progress monitoring have been used with success in elementary grades since the 1980s
- Elementary measures include examples of both curriculum sampling and robust indicators
- Several measures are available commercially as computer programs or Web-based systems

Elementary-Level Measures: Curriculum Sampling Approach

- Test items represent the critical skills in the grade-level curriculum (or represent grade-level state standards)
- Although administration time is held constant across the year, it may vary by grade level

Elementary-Level Measures: Curriculum Sampling Approach

- Measures may contain only computation problems or problems representing concepts and applications, or a combination of both
- Because the same skill types are tested repeatedly, analysis of student performance with respect to specific skills is possible

Developed Through Curriculum Sampling

Monitoring Basic Skills Progress: Basic Math

- Computation
- For Grades 1-6, test administration varies from 2-6 minutes, depending on grade level
- Scored as number of digits correct in answers (using specified scoring algorithms)

Random numerals within problems

Random placement of problem types on page

Measure taken from Monitoring Basic Skills Progress: Basic Math Computation (2nd ed.) (1998).

Monitoring Basic Skills Progress: Basic Math

- Concepts and Applications
- For Grades 2-6, test administration varies from 6-8 minutes, depending on grade level
- Scored as number of blanks correct
- Computer program provides skills analyses

Name _______________________________

Date ________________________

Test 4 Page 1

Applications 4

Column A

Column B

(1)

(5)

Write a number in the blank.

Write the letter in each blank.

1 week = _____ days

•

(A)

line segment

Z

•

•

line

(B)

K

L

(6)

Vacation Plans for Summit

School Students

•

•

(C)

point

M

N

Summer

ray

(D)

School

Camp

(2)

Look at this numbers.:

Travel

356.17

Stay home

Which number is in the hundredths place?

0

10

20

30

40

50

60

70

80

90

100

Number of Students

(3)

Use the bar graph to answer the questions.

Solve the problem by estimating the sum or

difference to the nearest ten.

The P.T.A. will buy a Summit School

T-Shirt for each student who goes

Jeff wheels his wheelchair for 33 hours

to summer school. Each shirt costs

a week at school and for 28 hours a week

$4.00. How much money will the

$ .00

in his neighborhood. About how many

P.T.A. spend on these T shirts?

hours does Jeff spend each week wheeling

How many students are planning to

his wheelchair?

travel during the summer?

How many fewer students are planning

to go to summer school than planning

(4)

to stay home?

Write the number in each blank.

(7)

3 ten thousands, 6 hundreds, 8 ones

To measure the distance of the bus

ride from school to your house you

would use

(A) meters

2 thousands, 8 hundreds, 4 tens, 6 ones

(B) centimeters

(C) kilometers

One page of a three-page measure for mathematics concepts and applications (24 problems total)

Measure taken from Monitoring Basic Skills Progress: Basic Math Concepts and Applications (1999)

Yearly Progress ProTM

- Web-based progress monitoring system
- Both computation and problem-solving items are included on each form
- Each test, Grades 1-8, is administered for 15 minutes
- Multiple-choice format (scratch paper allowed)

See http://www.mhdigitallearning.com

Yearly Progress ProTM

- Scored as number of problems correct (out of a total of 30)
- Provides skills analyses for class and individual students
- Program also contains instructional exercises by skill

See http://www.mhdigitallearning.com

Yearly Progress Pro: Sample screen taken from an instructional exercise but also illustrates how items are presented (one by one) on progress monitoring measure

See http://www.mhdigitallearning.com

YPP: Skills Feedback Across Class

Shows specific skills tested for algebra cluster at Grade 6

Green circle indicates mastery; yellow circle indicates partial mastery; red circle indicates skill is not mastered

See http://www.mhdigitallearning.com

Progress Monitoring in Mathematics

EdCheckup: Cloze Math

- Web-based progress monitoring system
- Robust indicator consisting of basic facts in addition, subtraction, multiplication, and division--80 problems administered for 2 minutes
- May select electronic scoring option or paper and pencil option

See http://www.edcheckup.com

Taken from http://www.edcheckup.com

AIMSweb

- Web-based progress monitoring system
- Measures are printed and administered to students
- Variety of measures for Grades 1-6:
- Basic facts by single operation or mixed operations (robust indicators)--score by correct digits in answers
- Mixed skills by grade level (curriculum sampling)--no skills analysis available; score by correct digits in answers or by correct digits in answers and critical processes (as indicated on answer key)
- Graphs of student progress are provided

See http://www.aimsweb.com

Sample AIMSwebBasic Facts Measures

Taken from http://www.aimsweb.com

Why Should Your School/District/State Implement Progress Monitoring?

- What efforts have you already made toward implementation of progress monitoring?
- What are your goals for implementation for next year?
- Goals for 3 years from now?

ACTION PLAN

What considerations do you need to address as you choose a progress monitoring system?

- What type of information do you hope to collect about student progress?
- What approach you will use?
- Scope of implementation (school, class, or grade level)?
- Resources?
- Time
- Money
- Personnel
- Technology
- How will teachers be trained and provided

with ongoing support?

ACTION PLAN

Your plan for implementation …

- With whom will you work?
- 1 student, small group, whole class, school, district
- What materials will you use?
- When will you administer the probes and graph performance?
- Day, time
- With whom will you discuss your data and interventions?
- Colleague, principal, consultant, school psych, SLP
- How will you use the data?
- How will you determine instructional interventions and their effectiveness?
- What will you do if you have questions?

Reference

- Fuchs, L. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, 188-192.

Additional Resources

Progress Monitoring Measures

- AIMSweb Web site: http://www.aimsweb.com
- Edcheckup Web site: http://www.edcheckup.com
- Monitoring Basic Skills Progress (Macintosh (OS 9) computer program available through http://www.proedinc.com
- Project AAIMS Web site (algebra progress monitoring measures and research results) www.ci.hs.iastate.edu/aaims
- Yearly Progress ProTM Web site: http://www.mhdigitallearning.com

Additional Resources

National Centers

- National Center on Student Progress Monitoring (NCSPM): http://www.studentprogress.org
- Research Institute on Progress Monitoring (RIPM): http://www.progressmonitoring.org

Download Presentation

Connecting to Server..