1 / 94

Improving Performance: The Five Step Process

Improving Performance: The Five Step Process. Mimi Lufkin: Chief Executive Officer National Alliance for Partnerships in Equity California: September 2-3, 2008. The Five Step Process. Document Performance Results. GOAL: Understand the problem completely before you seek solutions.

kaylana
Download Presentation

Improving Performance: The Five Step Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Performance: The Five Step Process Mimi Lufkin: Chief Executive Officer National Alliance for Partnerships in Equity California: September 2-3, 2008

  2. The Five Step Process

  3. Document Performance Results GOAL: Understand the problem completely before you seek solutions How do you analyze performance data? What questions should be addressed? What tools and methods can be used to present and analyze data? How should data quality problems be considered in analyzing data?

  4. Perkins Accountability Measures • Participation data • 4S1 & 4P1 in Perkins III • 6S1 & 5P1 in Perkins IV # underrepresented students participating in NT CTE Participation rate: all students participating in NT CTE • Completion data • 4S2 & 4P2 in Perkins III • 6S2 & 5P2 in Perkins IV # underrepresented students completing NT CTE Completion rate: all students completing NT CTE

  5. Data CollectionDisaggregation required in Perkins IV Special Population • Underrepresented gender students in a nontraditional CTE program • Single Parent • Displaced Homemaker • Limited English Proficiency • Individuals with a Disability • Economically Disadvantaged • Gender • Male • Female • Race/Ethnicity • American Indian or Alaskan Native • Asian or Pacific Islander • Black, non-Hispanic • Hispanic • White- non-Hispanic

  6. California Online Data Sources • Postsecondary • CCCCO Perkins III Data • CCCCO Perkins IV Negotiation Data • Secondary • Data Quest • Cal PASS (future)

  7. Other Data Sources • No Child Left Behind (NCLB) Report Card • National Center for Education Statistics (NCES) • District Enrollment / State Level • Other Sources???

  8. Recommended Analyses Trends • At least 2 yrs • Prefer 3-5 yrs Comparisons • State performance level • Best performer in state • Selected peer benchmark • Set your own benchmark • Site specific • Statewide • District • School/College • Programs

  9. Data Quality Concerns Alignment, Scope, Reliability, Time & Coverage • Alignment: Extent to which instruments accurately measure what they’re supposed to measure • How are participation #s generated? By teachers? Districts? • Scope: Breadth of measurement with respect to state-identified performance outcomes • Do districts measure single parents in programs?

  10. Data Quality Concerns Alignment, Scope, Reliability, Time & Coverage • Reliability: Consistency in conducting measurements • Do all districts define “concentrator” similarly? • Timing: When measurement is made • Do some districts measure at diff. times of year? • Coverage: Base of students included in measurement • Include all secondary students? Only concentrators?

  11. Perkins Consolidated Annual Report Data State District or College Program Level Analysis

  12. State Level Perkins Nontraditional Data Analysis

  13. Statewide Secondary and Postsecondary Participation and Completion Data

  14. Statewide Secondary and Postsecondary Participation and Completion Data

  15. Statewide Secondary and Postsecondary Data • What does the data say or suggest? • How has state performed in each measure? • What happened to the results over time? • What questions arise from this data?

  16. District or College Level Perkins III Analysis

  17. College Participation and Completion Data

  18. College Perkins III Nontraditional Participation and Completion Data

  19. College Level Data • How has the college performed on each measure? • How has the college performed as compared to the state average on each measure? • What happened to the results over time? • What questions arise from this data?

  20. College Level Perkins III and IV Data Worksheet Complete Section A and B

  21. Program Level Perkins III Analysis

  22. State Program Participation and Completion Data09 TOP Code- Engineering and Industrial Tech

  23. State Perkins III Nontraditional Participation and Completion Data09 TOP Code Engineering and Industrial Tech

  24. Program Level Data • What does the data say or suggest? • How has the state performed in each measure? • What happened to the results over time? • What questions arise from this data?

  25. College Program Participation and Completion 09 TOP Code- Engineering and Industrial Tech

  26. College Perkins III Nontraditional Participation and Completion Data09 TOP Code Engineering and Industrial Tech

  27. Program Level Data • What does the data say or suggest? • How has the college performed in each measure? • What happened to the results over time? • What questions arise from this data?

  28. Program Level Perkins III Data Worksheet Complete Section C and D

  29. Other Analyses with Perkins Data • You can do disaggregated analyses by: • Ethnicity • Special population • Suggested comparisons and analyses: • Look statewide (secondary & postsecondary) and at specific sites and in specific programs • Compare % to see if certain groups choose NTO CTE programs at higher/lower rates than their representation in CTE programs in general • Look at total numbers as well as percents. Be aware of the effect of small sample size

  30. Skill Attainment in Engineering and Industrial Technology

  31. Completions in Engineering and Industrial Technology

  32. Completion Caveats • Completions are tracked for “concentrators” - those students who are seriously committed to the program. • There may be many dropouts before students become concentrators.

  33. Placement for Engineering and Industrial Technology

  34. Employment Retention for Engineering and Industrial Tech

  35. Nontraditional Participation: Special Population AnalysisEngineering and Industrial Tech (09) - Region 6 data

  36. Worksheet Activity-case study • Read the first page of the case study • At the bottom of the sheet, you’re asked: • What performance data should they review to help them identify their problem? • Answer this through completing: • “Document Performance Results” section of “The Five Step Process Worksheet” • What data can you generate? Who would this task tell you information about? Where could you get it? • This data would help you identify initial problems about nontraditional program participation & completion

  37. SAMPLE WORKSHEET: Document Performance Results • Evaluate

  38. The Five Step Process

  39. STEP TWO Identify Root Causes

  40. Why Search for Root Causes? • Keep from fixating on the “silver bullet” strategy • Identify the conditions or factors that cause or permit a performance gap to occur • Two types of causes: • Direct cause (i.e. instructional practice) • Indirect cause (i.e. teacher training)

  41. How to Identify Root Causes • Search for most direct & highest impact causes • Employ a systematic evidence-based process • Draw on current research and evaluation • Formulate and test theories or hypotheses • Use multiple methods and data sources • Note, you’re likely to find multiple causes

  42. Phase 1: Identify Potential Causes • Review Research Literature • Review Program/Institutional Evaluations and Effectiveness Reviews • Conduct Focus Groups • Peer Benchmarking • Interviews & Surveys • Brainstorm

  43. Resources on CD • Survey Instruments • Destination Success • How to Conduct Interviews • How to Conduct Focus Groups

  44. Other Resources The New Look Self-Study Illinois Center for Specialized Professional Support

  45. Phase 2: Analyze and Evaluate Potential Causes Group Causes Into Two Categories: • Group 1: Causes Within Your Control • School scheduling • Classroom climate • Faculty awareness and capacity • Group 2: Causes Outside Your Control • Media representation • Family demographics

  46. Phase 3: Test and Evaluate Potential Causes Within Your Control Select root causes that: • Have the strongest theory and evidence to support them • Focus on direct causes of performance gaps • Address the most critical needs • Provide the best opportunity to have high impact on performance • Are supported by stakeholders who will help develop and implement solutions (See page 17 of the OVAE Guidebook)

  47. Review Research Summary Root Causes Tables • Chapter 6 Tables 16, 17, 19, and 20 from, “Improving Performance on Perkins III Core Indicators: Summary of Research on Causes and Improvement Strategies” • Authors: Robert Sheets, Mimi Lufkin, and David Stevens for the National Centers for Career & Technical Education • Available at www.napequity.org

  48. Review Research Summary Bias Literacy: A review of concepts in research on discrimination By Ruta Sevo, Ph.D. and Daryl E. Chubin, Ph.D.

  49. Root Causes • Career Guidance Materials and Practices • Access to and Participation in STEM • Instructional Strategies • Classroom Climate • Role Models • Early Exposure

  50. Root Causes • Curriculum Materials • Occupational Choice/Segregation • School Climate • Student Attitudes • Support Services • Student Isolation Based on Gender

More Related