Conversations About Accountability: Special Educators Views on the Impact of State Testing Programs

1. 1 Conversations About Accountability: Special Educators� Views on the Impact of State Testing Programs Lisa M. Abrams, PhD Virginia Commonwealth University March 17, 2005

2. 2

3. 3 Purpose of the Study Explore the impact of state-testing programs on the teaching of students with disabilities. Impact much less-understood for students with disabilities Current state-testing programs designed without students with disabilities in mind

4. 4 Test-Based Accountability Systems (1) Current landscape: Reflects a shift from basic skills to higher academic expectations Includes accountability mechanisms to motivate change Reform Model: Content standards Tests Performance targets Incentives

5. 5 Test-Based Accountability Systems (2) Two-tiered system: state and federal policies Federal policy-No Child Left Behind Act Requires: Achievement testing in reading, math and science for all students in grades 3-8, and 1 year in high school All students reach �proficiency� by 2014 States, districts and schools meet �Adequate Yearly Progress� goals Affects current special education policy; reflect emphasis on access, participation, and outcomes

6. 6 What does Ed Reform mean for Students with Disabilities? Compliance w/IDEA: inclusion and access to the general curriculum States must: Document participation and performance of SPED students on large scale assessments. Provide for participation by allowing for accommodations and alternative assessments. Report on their performance in the same detail and frequency as general education students.

7. 7 Impact of Testing on the Educational Process Positive outcomes: Greater emphasis on professional development and support Standards provide clarity and focus Test results provide useful information Increased emphasis on problem solving, writing across the curriculum Negative outcomes: Focus on tested content can lead to a narrowing of curriculum Increased time spent on test preparation Increased frustration, stress, and anxiety for students Undermine professional efficacy of teachers

8. 8 Research Questions How special education teachers perceive the impact of state-mandated testing programs on� � the educational experiences of students with disabilities? �their own professional experiences? How do these perceptions differ from those held by general education teachers?

9. 9 Data Collection Subset of a larger interview study 32 semi-structured interviews; conducted between 2000-2002 in three states 10-11 interviews with special educators per state Each state had a different type of accountability system

10. 10 Interview Topics Standards Content Instruction Assessment Students Use of test results Variables that emerged from a series of factor analyses conducted during the initial analysis of the national teacher survey data Items comprising the school climate, pressure on teachers, alignment, and value scales all asked teachers to indicate the extent of their agreement to a series of statements. School climate scale-includes statements about school atmosphere, teachers� expectations, students� affective responses to the state test (pressure and anxiety) ex item: �Students are under intense pressure to perform well on the state test.� Pressure on teachers: Scale is comprised of items that asked teachers about the pressure they feel to raise test scores from different individuals (e.g. district superintended, building principal, parents). Also asked teachers about the educational and professional impact of test-related pressure. For example, teachers� were asked to indicate the extent of their agreement to the statement �The state-mandated testing program leads some teachers in my school to teach in ways that contradict their own ideas of good educational practice.� Alignment scale: comprised of items that focus on teachers� perceptions of their state content standards and association of their classroom practices with the content and format of the state test. Example items include �The state-mandated testing program is based on a curriculum framework that all teachers should follow.� and �My tests are in the same format as the state test.� Value scale: Includes items that address teachers� perceptions of the value of the state test as an indicator of student achievement and a measure of school quality and general benefits of the state testing program. Example items include: �Score on the state test accurately reflect the quality of education students have received.� and �Teachers in my school have found ways to raise state-mandated test scores without really improving learning.� Impact on tested-content and classroom activities scales: comprised of items where teachers were presented with a series of content/areas and activities and were asked to indicate how the amount of time they spent on each activity had changed in order to prepare students for the state test. Options ranged from �increased a great deal� to �decreased a great deal� . Tested-content scale included instruction in tested areas, instruction in areas w/high-stakes attached for example. The classroom activities scale included enrichment school assemblies, field trips, and classroom enrichment activities. The last three scales, are comprise items that asked teachers to indicate the extent to which they felt the uses of test results for school, teachers/administrators and students were appropriate. Options ranged from �very appropriate� to �very inappropriate.� School accountability scale included uses of test results such as: hold the district accountable, award school accreditation. Teacher/Admin. Scale included uses such as awarding teachers/administrators financial bonuses, fire faculty/staff. Student accountability scale included uses of test results for decisions about placement and promotion (promote/retain students in grade/graduate students from high school).Variables that emerged from a series of factor analyses conducted during the initial analysis of the national teacher survey data Items comprising the school climate, pressure on teachers, alignment, and value scales all asked teachers to indicate the extent of their agreement to a series of statements. School climate scale-includes statements about school atmosphere, teachers� expectations, students� affective responses to the state test (pressure and anxiety) ex item: �Students are under intense pressure to perform well on the state test.� Pressure on teachers: Scale is comprised of items that asked teachers about the pressure they feel to raise test scores from different individuals (e.g. district superintended, building principal, parents). Also asked teachers about the educational and professional impact of test-related pressure. For example, teachers� were asked to indicate the extent of their agreement to the statement �The state-mandated testing program leads some teachers in my school to teach in ways that contradict their own ideas of good educational practice.� Alignment scale: comprised of items that focus on teachers� perceptions of their state content standards and association of their classroom practices with the content and format of the state test. Example items include �The state-mandated testing program is based on a curriculum framework that all teachers should follow.� and �My tests are in the same format as the state test.� Value scale: Includes items that address teachers� perceptions of the value of the state test as an indicator of student achievement and a measure of school quality and general benefits of the state testing program. Example items include: �Score on the state test accurately reflect the quality of education students have received.� and �Teachers in my school have found ways to raise state-mandated test scores without really improving learning.� Impact on tested-content and classroom activities scales: comprised of items where teachers were presented with a series of content/areas and activities and were asked to indicate how the amount of time they spent on each activity had changed in order to prepare students for the state test. Options ranged from �increased a great deal� to �decreased a great deal� . Tested-content scale included instruction in tested areas, instruction in areas w/high-stakes attached for example. The classroom activities scale included enrichment school assemblies, field trips, and classroom enrichment activities. The last three scales, are comprise items that asked teachers to indicate the extent to which they felt the uses of test results for school, teachers/administrators and students were appropriate. Options ranged from �very appropriate� to �very inappropriate.� School accountability scale included uses of test results such as: hold the district accountable, award school accreditation. Teacher/Admin. Scale included uses such as awarding teachers/administrators financial bonuses, fire faculty/staff. Student accountability scale included uses of test results for decisions about placement and promotion (promote/retain students in grade/graduate students from high school).

11. 11 Data Analysis Inductive & deductive process Codes were generated and applied to transcriptions using HyperResearch software Code frequencies were used to identify key themes

12. 12 Views on State Standards One-half to 3/4 ranged from neutral to positive Greater consistency, more focus Increased attention on critical-thinking skills More interaction between special and general education. Opinions were dependent on the extent of students� disabilities. For my kids who can fit into an inclusive classroom�those standards are expectations for them and their parents about what real life is all about�For my other children, who are severely disabled�those standards don�t mean anything to them. Nothing. We�re basically worried about their getting through the day. Do they know their address and phone number? Can they take care of themselves? (MI elementary SPED teacher) Before ed reform, the inclusive piece was definitely not there and kids were quickly referred out [of the regular classroom]. And thank god that�s changed. Ed reform has just really encouraged things to mesh together�its great and its also helped the special ed teachers to become more award of what the curriculum is all about. (MA elementary SPED teacher) #1: Greater consistency both within and across districts Quality of education was improved in math with greater emphases on problem solving, and on literacy skills with the emphasis of writing across the curriculum. Respondents also indicated that students would be more ready for employment when they graduate. #2: Elementary educators and some rural educators found some of the standards to be developmentally inappropriate. Resources includes both the materials to implement the standards as well as professional development opportunities necessary for understanding and implementation. Social studies standards (and tests) were singled out as being particularly problematic: #1: Greater consistency both within and across districts Quality of education was improved in math with greater emphases on problem solving, and on literacy skills with the emphasis of writing across the curriculum. Respondents also indicated that students would be more ready for employment when they graduate. #2: Elementary educators and some rural educators found some of the standards to be developmentally inappropriate. Resources includes both the materials to implement the standards as well as professional development opportunities necessary for understanding and implementation. Social studies standards (and tests) were singled out as being particularly problematic:

13. 13 Views on Accountability Dependent on stakes attached to testing program. More positive in KS than in MA. Positive views-inclusion, shared sense of responsibility Negative views-testing process, developmental appropriateness of the test Before [special ed students were included in the state assessment], we didn�t really have a sense of accountability�now I am aware that we need to get our students ready in those areas�I think it is good because the students, when they know that other kids are being tested�they wonder �why can�t we be taking this test also?� Now that they can take it, I think it helps them�in their mindset because they want to be like everyone else. (KS middle school SPED teacher) It [MCAS] doesn�t do anything but make the person or school that didn�t do well feel lousy�if you�re born an apple, you�re going be an apple, if you�re born an orange. You�re going to be an orange. We can�t all be Einsteins and that�s okay. We don�t have to compete to achieve�what makes the hierarchy think that there is going to be a successful generation coming up if they pass the MCAS, and what are they going to do with the kids who don�t pass? (MA elementary school SPED teacher)

14. 14 Views on the Test Main concern is the test is developmentally inappropriate Test scores are not a valid indicator of special ed students� achievement-they may know the information but can�t take the test. For special education students, the impact of the test is negative; mainly because they are not taught at the grade level that they are tested. The students cannot read the test by themselves...they are frustrated, so they either do nothing or bubble in anything. (MI Middle School SPED teacher)

15. 15 Impact on Classroom Practices Need to make instruction choices about teaching content or basic skills Maintained a balanced/mixed view; recognized the positive and negative effects It is necessary to expose them to a lot more information because they will be tested on it. You tradeoff instruction in the skills that they are lacking with instruction in the things you know they are going to see on the test. They do become exposed to and aware of, not necessarily proficient at, a wider range of information�but you are also taking time away from instruction in [basic] skills that they don�t have�whether they never really understood a vowel sound or they never really understood subtraction or whatever the skills are that they didn�t pick up at the right time�you lose out on time for intensive instruction in skill areas that they need in order to do broader based instruction and test preparation. So, there are positives and negatives. (KS, elementary school SPED teacher) Here we are reporting that the impact increased with stakes in the three states where we conducted interviews. We do not know if this finding would hold up in a larger study (or do we � is there corroborating evidence from the survey study?). Here we are reporting that the impact increased with stakes in the three states where we conducted interviews. We do not know if this finding would hold up in a larger study (or do we � is there corroborating evidence from the survey study?).

16. 16 Impact on IEP Process Concern that IEP objectives are not aligned w/state testing requirements Comment on expanding the academic scope of the IEP process to meet demands of tested grade levels. The objectives of the state test do not correlate well at all with special education students� IEP objectives. If I have a fourth grade student who is at a first grade reading level, making him take the fourth grade reading test is counterproductive to say the least. (MI elementary school SPED teachers) I know when I�m writing an IEP for a 3rd grader, I�m always thinking about what they are going to be doing at the 4th grade. We want to be sure to get in the IEP all of the accommodations that can be made because they are going to be hit very hard in the 4th grade�[where] they need to be within that regular classroom, and you provide support there instead of having a small group because there�s just so much application and they need to hear what the other kids are saying. (KS elementary school SPED teacher) Here we are reporting that the impact increased with stakes in the three states where we conducted interviews. We do not know if this finding would hold up in a larger study (or do we � is there corroborating evidence from the survey study?). Here we are reporting that the impact increased with stakes in the three states where we conducted interviews. We do not know if this finding would hold up in a larger study (or do we � is there corroborating evidence from the survey study?).

17. 17 Impact on Students More negative student effects reported in MA than in KS and MI: Labeled as failures Concerns about dropping out MA also reported more parental concerns than other states A lot of these students have a history of failing. I think they just see the MCAS as one more blow�knock them down one more time. (MA high school SPED teacher) I don�t think we do them any favors for the sake of that high school diploma. For most of my students, they are not going to pass the MCAS, and its been a devastating experience for them because they have a feeling of hopelessness. (MA high school SPED teacher) I listen to 9th graders�these kids are 14 years old and they say I�ll never pass so I might as well quit school�It sends the wrong message to kids. That because you have learning disabilities, you don�t count. You are not valuable in society. (MA high school SPED teacher) So, as in classroom impacts, reported negative and positive impacts on students appear to increase as stakes levels do Positive impacts reported by about � respondents in MA and centered on student motivation, primarily, as well as the test improving education. Of the latter, however, a greater proportion of MA respondents indicated that the test had a negative impact on a student�s education.So, as in classroom impacts, reported negative and positive impacts on students appear to increase as stakes levels do Positive impacts reported by about � respondents in MA and centered on student motivation, primarily, as well as the test improving education. Of the latter, however, a greater proportion of MA respondents indicated that the test had a negative impact on a student�s education.

18. 18 Conclusions and Implications Similar influences on classroom practices and professional experiences for special and general educators Unique challenges for SpEd teachers SpEd teachers more likely to perceive negative effects on students Achieving goals of special education policy

19. 19 For additional information: See Clark, M. et al. (2003). Perceived effects of state-mandated testing programs on teaching and learning: Findings from interviews with educators in low-, medium-, and high-stakes states. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College. Above report can be accessed at http://www.bc.edu/research/nbetpp/