Improving Impacts of Classrooms: Professional Development and Classroom Observation

1. Improving Impacts of Classrooms:Professional Development and Classroom Observation Robert C. Pianta, Ph.D. Dean, Curry School of Education Director, Center for Advanced Study of Teaching and Learning Discover. Create. Change.

2. Questions and challenges for policy, research, and training in education What experiences for social and cognitive development are offered to students in classroom settings? Do interactions with teachers and experiences in classrooms matter for students? Can observation leverage efforts to improve the richness, quality, and effectiveness of experiences in classrooms? Can we use observation of teacher-student interactions to improve student learning?

3. Student-teacher interactions and schools • Interactions with adults form “infrastructure” for school success: • Self-regulation, emotional self-control • Task orientation, persistence, motivation • Cognitive processes, language • Instruction is, in part, a social process: • Interactions with teachers are a “medium” for engagement • Good instruction is embedded in interactions • Interactions operate across content/curriculum

4. Can observation improve teaching? ® ® ® ® ® ® Scalable, reliable, valid, coarse distinctions – selection/retention Reliable, valid, articulated in experience of practitioners – tied to actual mechanisms of teacher impacts Consistent scoring across exceptional variation Technical properties support decisions and transparency Need for common lens/language Need for technical support and training infrastructure • HR policy • Program / professional development • Scale and Standardization

5. National-level observation studies National-level studies National Center for Early Development and Learning (NCEDL) NICHD Study of Early Child Care Up to 1,000 settings observed at preschool, K, 1, 3, 5 – more than 4,000 classrooms Large set of systematic standardized classroom observation in U.S. schools All teachers credentialed/certified

6. Observational/training procedures • Detailed manual describing codes/procedures • Master-coded video segments • Central training of observers live or web • Post-training practice and feedback • Videotaped certification test • Drift tests • Designed for typical day/classroom setup • Global ratings and time-sampled codes

7. Describing opportunities to learn: Counting behaviors, activities, practices • Vast majority of interaction/activity is whole group or individual seatwork • Few, if any, social or instructional interactions between teacher and individual child • Mostly literacy (50%-90% of instruction) • Exceptional variation within and across grades and classrooms • Consistent patterns from pre-k to 5th grade • No association with child outcomes

8. How do students spend time? • High-levels (30%) of “business/routine” activity • Pk-5: managing materials, routines • High levels of “basic skills” focus • 7:1 in pk-1; 14:1 in 3-5 • Ratio of listening, sitting, watching: Doing • 10:1

9. What is the CLASS? • CLASS is a tool for observing and assessing the quality of interactions between teachers and students • It measures the emotional, organizational, and instructional supportsprovided by teachers that have contribute to children’s social, developmental, and academic achievement. • CLASS is used to assess interactions among teachers and students for a variety of purposes: • Teacher Professional Development • Monitoring and Evaluation of Teacher Performance/ Effectiveness • Research

10. CLASS versions and development • Infant (CLASS-I) – presently in validation studies • Toddler (CLASS-T) – fully supported • Pre-Kindergarten (CLASS-Pre-K) – deployed at scale • Elementary (CLASS-K-3) – fully supported • Upper-Elementary (CLASS-4 to 6) – in validation • Secondary (CLASS-S) – in validation • Reliability/Validation/Training/Scale-Up

11. Classroom Organization Emotional Support Instructional Support Regard for student perspective Instructional learning Behavior management Positive climate Negative climate Concept development Quality of feedback Language modeling Productivity Sensitivity formats Relationships, Affect, Respect, Communication Clear expectation, Proactive, Redirection Analysis/reasoning, Creativity, Integration Punitive, Sarcasm/ disrespect, Negativity Feedback loops, Encourage responses, Expand performance Maximize time, Efficient routines and transitions Aware, Responsive, Address problem, Comfort Variety, Promote student interest, Clarity, Engaging Conversation, Open-ended, Repeat/extend, Advanced language Flexibility, Autonomy, Student expression DOMAINS DIMENSIONS INDICATORS

12. Classroom ratings: CLASS PK-5 Emotional Support • Positive climate • Negative climate • Teacher sensitivity • Regard for student perspectives • Effective behavior management • Learning formats/engagement • Productivity • Concept development • Evaluative feedback • Language modeling Organization/ Management Instructional Support

13. How is the CLASS Organized? Emotional Support Domain Dimension Indicator BehavioralMarkers

14. Data on CLASS • CLASS has been used to observe over 8,000 classrooms across the United States. • CLASS is one of the most extensively used observational measures of teacher effectiveness for preschool through secondary classrooms. • Classrooms are generally passive settings – 25% score quite low on all; 25% fairly high. Instructional Quality generally quite low on average • Teachers with higher scores on CLASS have students who make greater academic and social progress during the school year.

16. Profiles of classroom quality: First grade 7 6 5 4 Quality Emotional 3 Instructional 2 1 0 31% 23% 29% 17%

17. What observational studies tell us • Exceptional variability within and across grades, generally passive instructional environments • Little to no association of observed behavior: • Teacher experience or training, curriculum, public/private • Teacher salary • Small associations (.10 - .20) • Class size: larger classes more structured; smaller classes more social and higher instructional quality • Family income/education related to more positive ratings • Students needing access to stable high-quality instruction do not typically receive it – 10% rate

18. CLASS and student outcomes • Designs that isolate effects for instructional and emotional inputs controlling for other influences • Family and demographics, prior performance, teacher/school • Consistent results, small-moderate effects (.10 - .20) • Instructional and emotional quality predict more positive achievement and social outcomes • Larger effects on more proximal outcomes (e.g., child engagement) • Stronger effects for groups of children (~ .5 s.d.) • Low maternal education • Adjustment problems in K • Poor

19. Pre-k quality and growth in child outcomes • Where should we focus attention in policy, program development, and teacher preparation? • Predicting achievement growth during pre-k from: • Structural features (teacher ed., curriculum, etc.) • Observed interactions (ECERS, CLASS) • No association of structure with outcome, singly or in combination (e.g., NIEER index) • Instructional and Emotional Supports (CLASS predict positive changes in literacy, language, and math skills • Small effect sizes persist into kindergarten

20. Predicting student development in pre-k Changes in children’s development from beginning to end of preschool Mashburn, et al. (in press)

21. Do associations persist into kindergarten? - Burchinal et al., (in press) • Yes, children in pre-k classrooms offering higher levels of Instructional Support displayed better language skills at the end of the kindergarten year. • Kindergarten Instructional Support scores made an independent contribution to gains in children’s language and math abilities. • One-point differences in observed instructional supports appear linked to shifts in child outcomes.

22. “Active range” for effects • Analysis of “thresholds” – points on distribution where impacts are evident • Emotional Support – “4.5” and above • Instructional Support – “2” and above • For IS, the active range appears linked to teachers’ increased support for cognitive skills • One-point shift appears meaningful

23. Dual-language learners • Two major studies • Does CLASS describe interactions in language-diverse settings? • Does CLASS predict outcomes similarly? • CLASS 3-factor structure and psychometric properties same across diverse classrooms • Predicts outcomes similarly for DLLs • Evidence supports use in diverse settings

24. Gains in grade 1 achievement in instructionally supportive classrooms 107 106 105 Standardized tests of achievement adjusted 104 103 High educ. 102 101 Low educ. 100 99 98 Low Moderate High 1st Grade Instructional Support

25. Gains in grade 1 achievement in emotionally supportive classrooms 107 106 Kindergarten adjustment problems 105 104 Standardized tests of achievement adjusted 103 No problems 102 101 Multiple problems 100 99 98 Low Moderate High 1st Grade Emotional Support

26. CLASS-S results • Predicts state-standards test scores across all content areas (>2,000 students, >100 classrooms) • Moderate effect sizes (. 30) in all content domains • Also predicts observed student engagement • Accounts for gains in standards tests in the subsequent year • Now in studies with several thousand classrooms (MET, WTG study).

27. Support for high-quality interactions Professional development/ training Observational Assessment Social and academic outcomes for children CLASS Instructional Organization Social Resource allocation Improved teacher outcomes Evaluation Curriculum

28. Improving Caregiver-Child Interactions: MyTeachingPartnerCoaching, Video Library, Course

29. Aligned professional development: MTP • Connecting outcomes and inputs - Alignment • CLASS – specific definitions of interactions • Video Library – analysis of others’ interactions • Coaching – ongoing analysis/feedback on own interactions • Course – knowledge and analytic skills • All tested in RCTs

31. CLASS examples: PK-3

32. Video library: Secondary myteachingpartner.net myteachingpartner.net

33. Detailed video examples www.mtpsecondary.net myteachingpartner.net

34. MTP consultation cycle 1 Classroom video recording at an established time 2 4 Teacher and consultant meet and discuss teaching practices Consultant reviews and edits video clips 3 Teacher reviews clips and reflects on practice myteachingpartner.net

35. MTP Prompts: Feedback for teachers myteachingpartner.net

36. MTP Prompts: Feedback for teachers myteachingpartner.net

37. MyTeachingPartner research 240 participating pre-K teacherssplit into groups based on three conditions: Materials (traditional) Website access, Materials Website access, Materials, MTP Support high-quality teacher-child interactions with the CLASS as the frame

38. MTP Coaching improves interactions 5 4.5 MTP Teacher Sensitivity Control 4 3.5 May April June March January October February November December September

39. Classrooms with high poverty benefit most from MTP coaching for teachers 5.5 5 4.5 Coaching--100% Poor Teacher Sensitivity Control--100% Poor 4 3.5 May April June March October January February November December September

40. Findings: Effects of MTP support • Teachers with MTP coaches • Grew more sensitive in interactions with students • Increased students’ engagement in instruction • Improved language stimulation techniques • High-poverty classrooms benefit a great deal • Early career teachers benefit from coaching and video • Children with MTP teachers • Made greater gains in tests of early literacy • Experienced lower levels of problem behavior • Demonstrated higher levels of expressive language

41. NCRECE professional development study NCRECE evaluates two PD supports to teachers (N=450) : In-service course on effective support of language/literacy development Course improves CLASS Instr. Qual. In-service consultation using MyTeachingPartner

42. MTP in High School • Same approach – MTP coaching, video library • Randomized evaluation study – >100 classrooms • 6th-11th grades, all content areas • Teachers improved instruction; kids more attentive, engaged • Average student with MTP teacher improved 35th-60thpercentile on state high-stakes standards tests • Replicating results in another VA city

43. Scale-up – At this moment • CLASS in Head Start monitoring nationally (mandated in Federal law); 50,000 teachers • 4800 pre-k classrooms in GA • Statewide for Birth-5 programs in 2 states • 10,000 preschool classrooms in CA • 1600 preschool classrooms in Chicago • 17,000 4th-11th grades classrooms in Gates-funded MET study • Trained > 3,000 observers , 150 coaches • MTP – training local coaches

44. Continuing innovation • Video Assessment of Interactions and Learning (VAIL): Predict teacher performance before they set foot in the classroom. Hiring new teachers? • Web-based feedback systems to districts/states • On-line version of the MTP course • Teacher preparation programs, credentialing • Continued development of standardization and decision-making supports

45. Standardized observation of interactions • Feasible, reliable and valid at scale – A scalable language and lens for classroom settings p-3 • Three domains: Emotional, Organizational, Instructional appear valid across grades • Work in K-12 years confirming pk-3 work • Small increments, particularly in IS, matter for child outcomes • A focus for teacher professional development and preparation to increase quality and child outcomes

46. Accountability and program development • New policy frameworks (QRIS; HCMS; RTT) offer potential but also require careful evaluation • Accountability systems – wide-scale monitoring and feedback at correct level of aggregation. Link to aligned, proven-effective PD to target improvements in metrics. • Require capacity – # of observers/quality control • Live or video coding and stakes of the data • Issues to be resolved: cutpoints, combining with other metrics, implementation

47. Program development and improvement • Align measures with feedback to systems/individuals and resources to improve • Aligned PD can “move” interactions into “active range.” • Incentives and mechanisms to use effective PD • Degrees: Award credit for participation in effective PD. Certification: Award for competent performance.

48. Implications for P-12 • Standardized, valid observations can be scaled and integrated into human capital management and PD • Focus performance measures and PD on teachers’ interactions with children – common language and aligned systems • Aligned PD improves interactions and student outcomes – experimental studies • Incentives, supports, and targets for teacher behavior/performance in classrooms • We can improve teachers’ impacts on kids