CCSSO Large-Scale Assessment Conference Nashville, June 19, 2007

1. Middle School Math Instruction and Assessment for Students with LD, Students with MMR, & ELL Students:A Review of the Literature CCSSO Large-Scale Assessment Conference Nashville, June 19, 2007 Bob Dolan, Boo Murray & Nicole Strangman

2. Purpose • Comprehensive literature review of research-based practices during instruction and assessment of students with learning disabilities (LD), students with mild mental retardation (MMR), and students who are English language learners (ELL). • Instructional techniques include instructional approaches as well as scaffolds and supports used in the classroom. • Assessment techniques consider test design and delivery, with emphasis on testing accommodations. • Focus on identifying common approaches, despite large heterogeneity within each group. • Goal to support states in understanding how these students may be represented within a definition of “students in the gap.” • Only considering students who would take general assessment (i.e. not considering students who would qualify for AA-AAS).

3. Methodology • Focused on articles, reports, and conference presentations published between 1995 and 2006, including but not limited to peer-reviewed material. • Relevant articles were identified by browsing personal literature databases and organizational databases (e.g. NCEO), and by performing combined keyword searches of the online PsychInfo and Eric databases. • Search terms: “math” AND (“cognitive disabilities” OR “mild mental retardation” OR “English Language Learners” OR “limited English proficiency” OR “learning disability” • Searches expanded to automatically include related words. • Only articles reporting research that included middle school students (grades 6-8) or otherwise addressing topics of relevance to this population were included.

4. Included Student Populations • Mild Mental Retardation (MMR) • Students identified as having MMR, being “educable mentally retarded,” or described as having mental retardation and an IQ within the range of 50-55 to approximately 70 (DSM-IV). • Learning Disabilities (LD) • Students identified as having LD, a specific LD, or a reading disability as determined by the study author(s). • Students not diagnosed with a specific LD but having had low performance in math calculations that would presumably meet the IDEA definition for LD. • No attempt was made to evaluate the methodology used to diagnose LD (e.g. discrepancy model, response to intervention model); assumed that authors followed the standard IDEA definition of LD. • English Language Learners (ELL) • ELL students, English as a second language (ESL) students, and limited English proficient (LEP) students.

5. Students with MMR:Instruction • Shallow base of literature. • 4 articles • Shift from interventions focused on basic skills instruction toward interventions focused on computation and problem solving. • Example practices • Use of manipulatives to model word problems. • Strategy instruction combined with a concrete-representational-abstract teaching sequence. • Despite methodological issues (small sample, nonhomogenous age groups, lack of appropriate controls), studies suggest students with MMR to be active and strategic learners, able to learn to apply strategies in the context of math problem solving, and that this learning can result in improved mathematics computation and problem solving. • Findings also suggest that students with MMR may not transfer such learning to other problem-solving contexts or more difficult forms of problem solving, and that outside of explicit strategy instruction providing students with plentiful practice opportunities may help them to develop transferable strategies.

6. Students with MMR:Assessment • Research extremely limited. • Little information available regarding the effectiveness of assessment accommodations for students with MMR and their impact on the validity of inferences drawn from test results. • Helwig and Tindal (2003) addressed effectiveness of a video read-aloud accommodation for middle school students in various disability groups (MMR, n=12; LD, n=95; language impairments, n=5; emotional disturbance, n=10; other, n=8) and their peers without disabilities taking a mathematics achievement test. Because differences in the impact of the accommodation by disability category were not investigated, it is unclear whether students with MMR benefited from the accommodation.

7. Students with LD:Instruction • Research has addressed two general areas of mathematics instruction: basic skills and problem-solving. • Both areas have a sizeable research base; computation and related instruction and problem solving were identified as two major areas of research in Miller, Butler, & Lee’s (1998) review of the research on mathematics instruction for students with LD.

8. Students with LD:Instruction – Basic Skills • Two basic approaches: • Supporting students’ developing understanding of the number concepts underlying the skill, such as: • Teaching of underlying concepts as an integral part of learning basic facts and computational algorithms, such as concrete manipulatives. • Development of students’ ability to monitor and reflect on their work, through what may be referred to as self-regulation, self-instruction, or self-explanation. • Focus on fluency through practice and memory aids, such as: • Practice retrieval of addition, subtraction, and multiplication facts daily for 8-20 weeks. • Memory strategies to help recall of facts and procedures. • Both approaches have been found to be effective, although the sample sizes have been small. • Approaches not mutually exclusive; future research may help determine for which type of LD each approach is most beneficial.

9. Students with LD:Instruction – Problem Solving • The typical special education curriculum in mathematics focuses on basic computation skills, not the problem-solving skills that are essential for success in the general education mathematics curriculum (Cawley et al., 1998; Maccini & Gagnon, 2002) • Research base nonetheless suggests various effective methods for teaching problem-solving to students with LD, according to three basic approaches: • Use of specific schemata or organizers to facilitate problem representation, such as: • Graphic organizers. • Problem-representation schemata followed by problem-solution schemata. • Self-regulation to develop, monitor, and evaluate the results of a plan for solving a problem, such as: guided instruction. • Use of authentic, engaging problems that motivate students to find solutions, such as anchored instruction. • As with basic skills approaches, problem solving approaches can be complementary, and different types of students may more clearly benefit from different types of approaches.

10. Students with LD:Assessment • Although accommodations for students with LD are an important component of large-scale testing, there is limited research supporting the efficacy of accommodations and their impact on the validity of the inferences drawn from test results. • The read-aloud accommodation is the most commonly researched accommodation for this population, and results are contradictory. • Extremely limited research on pacing and computer-based testing. • No research found on extended time, memory supports, manipulatives, and calculators as accommodations. • Classroom-based assessment • Curriculum-based measurement / progress monitoring with concept and application probes is an effective means to improve LD students’ math performance.

11. ELL Students:Instruction • Four articles published between 1995 and 2006 on the topic of middle school mathematics instruction for ELL students. • Studies suggest difficulties with mathematical language could undercut mathematics performance both directly and indirectly by necessitating time-consuming strategies such as translation and rereading, which could slow progress through the test. • Two emergent themes: • Importance of fostering mathematical conversation. • Importance of building connections between mathematics and ELL students’ personal lives and cultures.

12. ELL Students:Assessment • Validation of instructional literature by identifying a mixture of linguistic, cultural, and mathematical factors that influence the test performance of ELL students, and the linguistically, culturally, and socioeconomically diversity of these students. • Specific language-related difficulties include unfamiliarity with vocabulary, literal interpretation of content under inappropriate circumstances, poor reading fluency leading to slow progress through the test, and difficulties understanding the syntax used (Abedi, Courtney, & Leon, 2003). • Fairly extensive documentation of a persistent gap between ELL and non-ELL student performance as well as evidence to support construct-irrelevant language issues as a causative factor. • Language-related factors have received the most attention in the assessment literature. • Support for use of dictionaries and glossaries but concern over threats to validity. • Use of native language testing, dual-language testing, and linguistic modification inconclusive.

13. Conclusions • Large disconnect between the level and types of instructional supports and testing accommodations for all three student populations. • Instructional supports focus largely on pedagogical approaches toward reducing barriers to learning. • Test designs, modes of administration, and accommodations largely limited to reducing accessibility barriers. • Discontinuity reflects the limited nature of current large-scale assessment techniques and psychometric approaches toward their design. • Concern over compromising the validity of test inferences or comparability of scores of students who do and don’t receive such supports. • General failure to consider the heterogeneity of the student population that could significantly impact the effectiveness of test design factors, modes of administration, and accommodations. • As a result, techniques that may be largely successful in allowing these students to learn effectively may not available at the point that students must demonstrate learning.

14. Considerations for Future Research • Approaches toward assessment that better dovetail with the supports students receive instructionally. • Additional research focused on methodologies for test development that consider construct-relevant vs. construct-irrelevant factors, such as Evidence Centered Design (Mislevy et al.). • Approaches that consider individual student differences, such as through application of universal design (Mace et al., 1996) and Universal Design for Learning (Rose & Meyer, 2002) principles, to create and administer tests that consider diverse students from the start (Thompson et al., 2002) and flexible tests that include built-in supports for diverse students (Bryant & Rivera, 1997; Dolan & Hall, 2001; Dolan, Hall, Banerjee, Chun, & Strangman, 2005; Ketterlin-Geller, 2005).