Improving Teacher Preparation: Including Knowledge and Skills on Evidence-Based Practices

1. Improving Teacher Preparation:Including Knowledge and Skills on Evidence-Based Practices Dan Reschly, PhD & Lynn Holdheide, MA Vanderbilt University/National Comprehensive Center for Teacher Quality Martha Hougen, PhD The University of Texas at Austin Kelley Regan, PhD George Mason University OSEP Project Directors Meeting July 20, 2011 12:30 p.m. - 3:00 p.m.

2. The Push to Refocus…Teacher Evaluation • Research, Papers, and National Efforts (e.g. The Widget Effect, So Long, Lake Wobegone) • All teachers are rated as good or great. Because of this… • Excellent performance goes unnoticed • Typical goes without support to improve further • Chronically low performing goes unaddressed • Results of Teacher Evaluation have little/no impact on HR decisions • Retention, promotion, placement, compensation, professional development, tenure, etc. • Policy • American Recovery and Reinvestment Act & Race to the Top (RTTT) Priorities Slide courtesy of Tony Bagshaw, Battelle for Kids

3. The Push to Refocus…Teacher Preparation • 1998 Higher Education Act Reporting Requirements • Since 2001, 27 states have never identified a single teacher preparation program as substandard • Since 2002, 119 institutions have been named at-risk or low performing. 58 are repeat offenders and 115 are still approved by their state • Few consequences for those that are named • Research has found no difference in student achievement outcomes of teachers from accredited or non-accredited programs (Analysis by Education Sector: https://title2.ed.gov/default.asp; Aldeman, et al., 2011) • Potential ESEA & HEA Reauthorization Recommendations

4. Addressing Teacher Quality in Teacher Preparation • Adequate teacher preparation building blocks to increasing the quantity and quality of the teacher force • Increasing teacher effectiveness and equitable distribution are core reform efforts. Federal funding supports efforts to • Provide quality instruction • Well-crafted learning experiences • Sustained implementation regarding evidenced-based practices

5. Obama Administration….A Push Toward Accountability in Teacher Preparation • Overhaul reporting requirements for higher education • Presidential Teaching Fellows • Scholarship Funding for high-quality teacher candidates • Distinguishing between exemplary and lackluster programs and routes • Three measures of accountability • Where graduates are placed & how long they stay in the profession • How much graduates help the students learn • If employers are satisfied

6. State Accountability for Teacher Preparation within RTTT • Louisiana • Uses VAM to calculate how well students taught by 1st and 2nd year certified teachers from specific program performed compared to similar students by experienced teachers • Teacher preparation programs are placed into one of five levels • Tennessee • Teacher Preparation Program Effectiveness Report Card • TN Higher Education Commission and stakeholder group will work to reward successful programs and support and/or decertify others

7. State Accountability for Teacher Preparation within RTTT • Florida • Longitudinal database links students to their teachers and courses, and teachers to teacher preparation programs and professional development • Not currently used in the approval process, but publically reported. Plan to use if for approval process based on: • Impact on student achievement • Meeting LEA and state needs for new and retrained teachers • Program’s contribution to induction and professional development. • Teacher Advisory Council to identify “Accomplished Practices” for both teacher evaluation and teacher preparation

8. State Accountability for Teacher Preparation within RTTT • Georgia • Teacher Effectiveness Measure (TEM), Leadership Effectiveness Measure (LEM), and District Effectiveness Measure (DEM). • By July 2011, TEMs and LEMs will feed into a Teacher Effectiveness Program Effectiveness Measure (TPPEM) • TPPEM • 50% student achievement of graduates • 20% of success rate of induction teachers • 20% content knowledge of graduates • 10% persistence of graduates

9. Among the local administrators, reported that contractual agreement prevented modification in the evaluation process. Modification of Evaluation Processes for Special Educators 81%

10. Evidence-Based Practices • Meeting the needs of “diverse” learners may not attend to the following: • Special skills (individualized education program [IEP] facilitation, collaboration, secondary transition, social and behavioral interventions, compliance with legal mandates) • Evidence-based instructional methods (direct/explicit instruction, scientifically based reading instruction, learning strategy instruction)

11. Opinions Regarding Special Education Teacher Evaluation Strongly Agree or Agree 92% 32% 84%

12. Teacher effectiveness, equitable distribution, and teacher preparation are inextricably linked. • TQ Center developed Innovation Configurations to encourage examination of the strengths and gaps within programs to answer the following questions: • What types of instruction and experiences do teachers receive throughout their preparation and/or professional development that promote the use of evidenced-based instructional practices? • To what extent are teachers and teacher candidates provided an opportunity to apply these strategies with explicit feedback and sustained implementation and support to ensure fidelity?

13. Workshop Overview • Purpose • To familiarize participants with tools designed to improve teacher preparation and professional development practices related to scientifically based instruction and classroom organization and behavior management… leading to improved teacher qualifications and enhanced student outcomes • Outcomes • Participants will be able evaluate course syllabi by using research-based practices as a guide • Positive results on the OSEP Personnel Preparation Program indicators

14. Innovation Configurations • Hall & Hord, 1987; Roy & Hord, 2004 • Innovation configurations. Matrix specifying • Essential components • Level of implementation • Vanderbilt University’s innovation configurations content based on • Research • Policy • Best Practices

15. What is an Innovation Configuration? • Used for more than 30 years in development and implementation of educational innovations and methodologies. • Evaluate programs • Evaluate fidelity of implementation of educational interventions • Most commonly, professional development tools (i.e., guide implementation of innovation within a school and to facilitate the change process)

16. Application of Innovation Configurations • IHE faculty self-assessment, self-reflection, course improvement • IHE department heads, deans, and other university administrators interested in ensuring high-quality instruction in teacher preparation programs • State departments of education seeking to unify instruction statewide with common language and goals consistent with federal policy (e.g., Maryland and Colorado) • Design of professional development • Professional association standards • State licensure and teacher education program approval requirements

17. Purpose of Innovation Configurations (ICs) Purpose of TQ Center ICs • To evaluate and improve teacher preparation coursework and continuing professional development --- • focusing on the degree to which federal policies and Scientifically Based Research (SBR) are implemented in coursework and supervised experiences in reading, classroom behavior management, inclusive practices, and learning strategies

18. Innovation Configurations • Barriers to the Preparation of Highly Qualified Teachers in Reading (http://www.tqsource.org/publications/June2007Brief.pdf) • Effective Classroom Management: Teacher Preparation and Professional Development (http://www.tqsource.org/topics/effectiveClassroomManagement.pdf) • Teacher Preparation to Deliver Inclusive Services to Students With Disabilities (http://www.tqsource.org/publications/TeacherPreparationtoDeliverInclusiveServices.pdf) • Teacher Preparation and Professional Development in Effective Learning Strategy Instruction (http://www.tqsource.org/publications/EffLearnStrtInstructionIssuePaper.pdf) • Teacher Preparation for Response to Intervention in Middle and High Schools (http://www.tqsource.org/publications/IC_RTI.pdf) • Preparation of Effective Teachers in Mathematics (http://www.tqsource.org/publications/IC_Mathematics.pdf) • Linking Assessment and Instruction: Teacher Preparation and Professional Development (http://www.tqsource.org/publications/EffLearnStrtInstructionIssuePaper.pdf): • Upcoming • Direct and Explicit Instruction • Data Use and Interpretation by the Classroom Teacher

19. What Works? See Kavale (2005, 2007), Learning Disabilities, 13, 127-138 and other sources TreatmentEffect Size • Applied Behavior Analysis (many applications) + 1.00 • Formative evaluation: CBM+ Graphing+Decision Rules+ Reinf. + 1.00 • Explicit Instruction and Problem Solving + .70 to 1.50 • Learning & Comprehension Strategies +1.00 • Math Interventions +.60 to 1.10 • Writing Interventions +.50 to .85 • Matching instruction to learning styles?? 0.00 • Note, these effect sizes are stable across cultural groups

20. What Does NOT Work (Forness et al, 1997; Kavale 2005, 2007) • Perceptual motor training • Matching instruction to presumed cognitive strengths • Training cognitive weaknesses (e.g., working memory) to improve achievement • Special classes for students with high incidence disabilities (exceptions?) • Unstructured instruction with learners who have limited prior knowledge • Which do we see more of in current teacher preparation? Reschly RTI

21. Some things do not make sense

22. Innovation Configurations • Tool used to evaluate teacher preparation coursework through reviewing syllabi • Specifically assess the degree to which evidence-based practices are implemented in required courses • Focus on entire program, not a single syllabus • Established through tables with two dimensions: • Key essential components • Levels of implementation

23. Research on Reliability • Vanderbilt University conducted a study of syllabi from 26 of 31 special education teacher education programs in large-population state using the three ICs (Reschly, Holdheide, Smartt, & Oliver, 2007) • Both exact ratings from two independent judges and one plus or minus were obtained for interjudge reliability. Scores across the three ICs ranges from approximately .79 to .85, well within the sufficient range for program evaluation purposes.

24. Use of Course Syllabi • Common practice across disciplines • Syllabus is a contract • Examine all courses in teacher education regarding IC components, not just a single course • Limitations of Syllabi • Incomplete reflection of course content and activities • Difficult to judge depth of experiences • Some content on syllabus not taught and some content is taught that is not on the syllabus • Overall: Syllabi reflect major features of program

25. Teacher Preparation • Significant gaps in the preparation of general and special education teachers • Scientifically-Based principles for both, including randomized controlled trials (RCTs), often not applied • Gaps in: • Teacher preparation and practice • State program approval and teacher licensure • Professional association standards

26. Summary: Teacher Preparation and Practice • Insufficient use of evidence-based practices in teacher preparation/comprehensive professional development and practice • TQ Center use of evidence-based innovation configurations to address these issues • Improved implementation of evidence-based principles leading to improved outcomes • Major Challenge: Narrowing the gap between what is known about evidence-based instruction and teacher preparation and special education practice

27. Classroom Organization and Behavior Management • Oliver, R. M., & Reschly, D. J. (2007). Improving student outcomes in general and special education: Effective classroom management. Washington DC: National Comprehensive Center for Teacher Quality. • Rationale: Teacher attrition, student achievement, improves inclusion outcomes. • Most beginning and some experienced teachers report significant problems with “discipline.” • Improved behavior also critical to ESEA and IDEA outcomes.

28. Components of Good COBM • Structured environment • Active supervision and student engagement • School-wide expectations • Classroom rules • Classroom routines • Encouraging appropriate behavior • Discouraging inappropriate behavior • Monitoring student behavior and adjusting plan • (Emmer & Stough, 2001; Kerr & Nelson, 2002; Lewis & Sugai, 1999; Martella, Nelson, & Marchand-Martella, 2003)

29. Results of COBM IC Review

30. Implications of Findings • More emphasis on reactive vs. proactive strategies found in course syllabi • Greater emphasis on proactive may improve inclusive practices for students at-risk for or identified with EBD • Special educators should have a higher level of preparation in classroom organization and behavior management therefore general education teacher preparation may be a concern • Teacher preparation should include supervised practice to develop proficiency • Lack of entire course devoted to classroom management indicates slivered fashion of teaching content

31. Scientifically-Based Reading • Smartt, S. M., & Reschly, D. J. (2007). Barriers to the preparation of highly qualified teachers in reading (TQ Research & Policy Brief).Washington, DC: Learning Point Associates, National Comprehensive Center for Teacher Quality. • Rationale: Too many students read below basic levels • Many teacher education and professional development programs do not implement the scientifically based research on reading

32. National Assessment of Educational Progress 4th Grade Reading (2009) Tables A-12, A-15, & A-16

33. NAEP (2009) 8TH Grade Reading (2009) Tables A-20, A-23, & A-24

34. General Instructional Principle • Instruction at student’s knowledge/skill level Lower Prior Knowledge Higher Prior Knowledge Needs Complete, Explicit Systematic Can Profit from Incomplete Implicit Less Structured Reschly RTI Reschly RTI 34 34

35. What Is the Scientifically Based Reading Instruction - IC? • Essential Key Components (Content Validity) • Preventing Reading Difficulties in Young Children (Snow, Burns, & Griffin, 1998). • Teaching Children to Read: An Evidence Based Assessment of the Scientific Research Literature on Reading and Its implications for Reading Instruction (National Reading Panel, 2000). • Policy support • ESEA (NCLB, 2002) • IDEA (2004, 2006)

36. Essential Components • Scientifically-based reading research/NCLB/IDEA • Phonemic awareness • Phonics • Fluency • Vocabulary • Comprehension • Integration of components • Systematic and explicit instruction • Screening and progress monitoring assessment Reliability – approximately.85

37. KTG: Initial Sound Fluency Fall to January 05-06 Yr. Benchmark: Winter KTG 25 sounds correct/min. New KTG Teacher and Traditional Instruction

38. KTG: Initial Sound Fluency Fall to January 05-06 Yr. Benchmark: Winter KTG 25 sounds correct/min. Experienced Teacher Direct Instruction

39. Phoneme Seg. Fluency: Jan to May 05-06 Yr. Benchmark: 35 correct New KTG Teacher and Traditional Instruction

40. Phoneme Seg. Fluency: Jan to May 05-06 Yr. Benchmark: May 35 per minute Experienced Teacher Direct Instruction

41. Purposes of Universal Screening • Assess success of instructional program • Percent of students at or above benchmarks • If necessary, examine curriculum, instruction, or both • Identify students below benchmarks • Interventions within general education classroom • Assess progress and consider need for more intensive interventions at Tier II

42. Second Grade Oral Reading Fluency Benchmark: End of 1st=42 WCM Winter=71 WCM End of 2nd=100 WCM ?? Students needing greater Gen’l Ed monitoring and Interventions

43. Teacher Preparation • Significant gaps in the preparation of general and special education teachers • Scientifically-Based principles for both, including randomized controlled trials (RCTs), often not applied • Gaps in: • Teacher preparation and practice • State program approval and teacher licensure • Professional association standards

44. Using IC Data • How are data collected? (handout- steps, procedure) • How are ICs scored? (handout- steps, procedure) • How are data analyzed? (handout- steps, procedure)

45. Application of Data How are findings used – local level? • IHE administrators may conduct needs assessment throughout all required early reading courses across departments (e.g., teaching and learning, special ed) university wide • Lesson planning guide at the elementary and higher education reading course level to ensure that all elements of scientifically- based reading research are taught

46. Application of IC Data • How are findings used—policy level? • Evaluation and improvement of teaching preparation and professional development in reading • Examination and improvement of scientific-professional association standards for teacher preparation • Improved state teacher licensure standards and teacher preparation program approval

47. Communication • How can the findings be shared with educators and other stake holders?

48. Teacher Preparation • Significant gaps in the preparation of general and special education teachers • Scientifically-Based principles for both, including randomized controlled trials (RCTs), often not applied • Gaps in: • Teacher preparation and practice • State program approval and teacher licensure • Professional association standards

49. Key Essential Components • Problem areas in teacher performance • Content Validity based on • Authoritative research review and integration sources (e. g., Nat’l Rdg and Math Panels) • Research (Experimental research confirming the validity of the components for improving achievement) • Policy (NCLB and IDEA) • Listed on the left hand side of the IC • Descriptors and examples to guide review