Diocese of Fort Worth Curriculum Development Process Professional Development Evaluation Report

1. Diocese of Fort WorthCurriculum Development ProcessProfessional DevelopmentEvaluation Report • EDU: 8315-40 • Dr. Ballenger • Authors: • Pamela Cooper, Charlene Hymel, • Kary Johnson, Michael Wright

2. Executive Summary • Evaluation Questions EQ1: To what extent are teachers implementing the new standards-based curriculum? EQ2: What impact did the Curriculum Initiative staff development have on your design & implementation of the Year-Long Plan (YLP)? EQ3: What changes have you seen in teachers’ lesson planning based on implementation of Year-Long Plans and the Curriculum Initiative?

3. Executive Summary • Summary of Findings In general, the Curriculum Initiative staff development program has been successful in providing more opportunities for teachers to plan together leading to a more organized and detailed way to design lesson plans which tended to affect positive changes in student learning.

4. Executive Summary • Implications No response from mission/urban schools. Possible causes: lack of time for teachers, or disparity of resource allocation between socio-economic areas

5. Executive Summary • Recommendations It is advisable that future training sessions occur on individual campuses in an effort to target implementation strategies and to affect change in teacher behavior by grade level-grouped campuses, such as by all elementary schools or by all middle or high schools.

6. Introduction • Purpose of the evaluation is threefold: to review the extent to which teachers are implementing the new standard-based curriculum; to measure the impact of the Curriculum Initiative staff development on design and implementation of the Year-Long Plan (YLP); and, to validate changes in teacher lesson-planning based implementation of YLPs and the Curriculum Initiative.

7. Introduction • Goal of the evaluation: to evaluate whether grade level teachers have implemented a Year- Long Plan (YLP).

8. Introduction • Evaluation Questions EQ1: To what extent are teachers implementing the new standards-based curriculum? EQ2: What impact did the Curriculum Initiative staff development have on your design & implementation of the Year-Long Plan? EQ3: What changes have you seen in teachers’ lesson planning based on implementation of Year-Long Plans and the Curriculum Initiative?

9. Overview of the Program • Program description: The Diocesan Curriculum Development Process Staff Development Program moves teachers from check-off curriculum lists to standards-based curriculum. Drs. Ozar and Mia conducted staff development on year-long plans, essential learning, backwards design lessons, formative and summative evaluation, and instructional strategies over a three year period. • Content: Following a book study, staff development included whole group sessions, clustered school sessions, and sessions for the curriculum learning team members. Teacher learning teams were formed at each school to help facilitate continuous implementation and learning.

10. Overview of the Program • Program goal: implement standards-based curriculum, outcomes-based instruction to facilitate an academic program distinguished by rigor and continuous, sustained growth for students and teachers. • Objectives: (1) enable teachers to translate standards into school level curriculum (2) improve classroom instruction, and (3) increase student and teacher learning • Activities: identify essential learning, make year-long plan, match assessments to essential learning, select instructional strategies, design lessons in backward design to support instructional units, use learning results to inform instruction, and engage in PLC process

11. Overview of the Program • Resources: Hartford Curriculum Guides Texas Essential Knowledge & Skills Dr. Lorraine Ozar Dr. Michelle Lia A+ Educators Notebooks & Handouts Curriculum Learning Team Members Diocesan Office

12. Overview of the Program • Stakeholders: Pastors, School Advisory Councils, Administrators, Teachers, Parents, Students • Participants: Superintendent, Associate Superintendent, School Administrators, Teachers

13. Evaluation Design • Methods Mixed-Methods: Quantitative and Qualitative methods used in the form of surveys, focus-group interviews, and one-on-one interviews.

14. Evaluation Design • Data Collection - Triangulated Surveys Focus-Group Interviews One-on-One Interviews

15. Evaluation Design • Data Sources Survey responses from K-8 Teachers on YLP Checklist Focus-Group Interviews with teachers One-on-One Interviews with administrators

16. Evaluation Design • Data Analysis • Quantitative Analysis: • Descriptive statistics: conducted on (N=132) responses to 13 question YLP checklist survey • Inferential statistics: • 4 separate ANOVAs conducted • 4 independent variables: endurance, leverage, readiness, combined (e+l+r) • 3 existing groups/subject variables: grade level, subject, school

17. Evaluation Design • Data Analysis Qualitative Analysis: Interview responses were studied for patterns. Once patterns were found in the responses, commonalities were extrapolated and through analysis, formed a “picture” of program success.

18. Evaluation Design • Data Analysis • Descriptive Data (Means/SD): • response rate: • 38% of total K-8 faculty • 40% of total schools • Data analyzed by: • school (6), • grade (elementary or middle), • subject (ELA, math, science, SS, religion, specials, foreign language) • Full survey: 83 % “yes” elementary & 84% “yes” middle • See table 1 for more info on descriptive trends

19. Evaluation Design Data Analysis continued • Quantitative Variable 1: full survey • No significant differences between school, grade level or subject • All groups reporting endurance scores of 89-100% (table 1) • See table 2 for more information

20. Evaluation Design Data Analysis continued • Quantitative Variable 2: endurance • No significant differences between schools, grade levels or subjects • All groups reporting endurance scores of 89-100% (table 1) • See table 2 for more information

21. Evaluation Design Data Analysis continued • Quantitative Variable 3: leverage • No significant differences between schools or grade levels • Significant difference (p=.015) between subjects – issue with specials (79%) /foreign language (50%) as compared to core subjects (84-100%) • See table 3 for more information

22. Evaluation Design Data Analysis continued • Quantitative Variable 4: readiness • No significant differences between school, grade level or subject • All groups reporting endurance scores of 84-100% (table 1) • See table 4 for more information

23. Evaluation Design Data Analysis continued • Quantitative Data Set 5: combined (e+l+r) *only summative/outcome portion • set level of response 66.66% (by Dr. Ozar) • all stakeholder responses between 83-100% • significant differences between responses when analyzed by subject (p=.015) & school (p=.004) • See Table 5 for more info

24. Findings • Qualitative Interpretations: Broad themes emerged in the data • “Organization” which led to more detailed plans • “Collaboration” which led to the development of PLCs.

25. Findings • Quantitative/Qualitative Interpretations • Standard Based Curriculum Implementation is Occurring (EQ1, EQ3) • Continued Inconsistencies among Schools/Populations (EQ1, EQ2) • Non-Core Educators (EQ1) • Professional Learning Community Creation (EQ1, EQ2) • Improved Professional Development Delivery (EQ2, EQ3)

26. Findings • Delimitations • YLP checklist tool • Group (school, subject) size inconsistency

27. Findings • Limitations • Lack of diversity among reporting schools • Lack of diversity among teacher subject-areas • Lack of consistency among information disseminated during training sessions

28. Findings • Implications: lack of teacher response from mission schools • Possible reasons lack of time lack of resources

29. Recommendations • Future Actions • Continue Alignment Process • Promote alignment of Programming (school buy-in) • Reorganize delivery method of staff development (PLCS within schools) • Target-train based on grade level-grouped needs • Target Diversity/Mission Schools

30. References • See original report • Upon request