1 / 26

Quality Matters 2008

Quality Matters 2008. “We’ve got the Data Reports, now how do we use them?” -- UWM’s Data Feedback Loop UWM Team: Alison Ford Kerry Korinek Barbara Bales. Some Background, then Three Parts. I. What Data (8 Annual Reports)

Download Presentation

Quality Matters 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Quality Matters 2008 “We’ve got the Data Reports, now how do we use them?” -- UWM’s Data Feedback Loop UWM Team: Alison Ford Kerry Korinek Barbara Bales

  2. Some Background, then Three Parts I. What Data (8 Annual Reports) II. Using the Data (Data Feedback Loop) III. Questions/Sharing Your Strategies

  3. Some Background The Council on Professional Education (CPE) & its Steering Group Oversees the Assessment System --

  4. Some Background • We have 34 programs [60 licenses] across 5 schools/colleges • The CPE has voting members representing each program, in addition to some other appointments (e.g., an Associate Dean from Letters & Science and 2 program graduates) • The CPE meets 4x/year and usually has a couple of workgroups each year. Assessment process is overseen by our Council on Professional Education (CPE) & its Steering Group

  5. Some Background (continued) • The Team meets every three weeks • Designated offices/individuals take the lead on collecting and compiling data for assigned reports • Reports are written by the Assessment and E-Portfolio Coordinator and the individual(s) who collect and compile the data– the reports include summary findings for CPE discussion • The CPE Steering Group reviews reports before they are presented to the CPE The CPE Data Management Team compiles the reports under the leadership of the “Assessment & E-Portfolio Coordinator”

  6. I. What Data? • 8 Annual “Unit” Reports (1) Exit Survey Report (2) Standards/Portfolio Assessment Report (3) Follow-up Report - Graduates; 1-2 Years Out (odd years) - Graduates; 4-5 Years Out (even years)

  7. 8 Annual “Unit” Reports (Cont.) (4) Candidate Status Report (5) Praxis I Report (6) Praxis II Report (7) Field Experience Report (8) Job Location Report

  8. 1) Exit Survey Report • How do completers assess the extent to which they… • Have been prepared to meet standards • Know their subject areas & draw upon coursework in Letters & Science & the Arts • Are prepared for the first year in the profession (a grade is given) & whether they would choose the same certification program … And, other areas: Urban Education/Equity Mission, Field Experiences, Support Services & Employment Prospects

  9. 2) Standards/Portfolio Report • At what level of proficiency do faculty members rate completers (using the program’s standards rubric)?: • Proficient • Emerging Proficiency • Lacks Evidence of Proficiency • How do candidates self-assess (or programs assess) standards related to: • UWM’s Core Guiding Principle of Urban Ed/Equity (critically reflective & responsive about diversity & equity; aware of commitment & need for persistence; a learner & collaborator)? • Interpersonal Communication & Routine Use of Technology?

  10. 3) Follow-up Report • 1-2 Years Out (odd years)… Our graduates: • Where are they working? • How satisfied are they with teaching/selected profession? • How well prepared are they for the standards, their first years of working… and how do employers view their preparation? • 4-5 Years Out (even years): • Graduates let us know what they are doing; they give us information about professional development, accomplishments, impressions of our program, their plans, and some demographic information

  11. 4) Candidate Status Report • Who is admitted? • Who completed our programs? • Who withdrew or was discontinued & why? [This report includes numbers and demographics for each major question.]

  12. 5) Praxis I Report • What are the PPST pass rates for UWM candidates? • How do scores break down by gender, race/ethnicity, and age group? • How many waivers are there by subtest? and by program? • What is the relationship of those receiving Praxis I waivers to their pass rates on Praxis II?

  13. 6) Praxis II Report • What are the Praxis II pass rates for UWM candidates & by program? • How do scores break down by gender, race/ethnicity, and age group? • What is the passing percentage based on attempts? • Which tests have pass rates of 95% or below? • What is the breakdown of test categories for tests of concern?

  14. 7) Field Experiences Report • How many placements are made, what type (e.g., On-the-Job), and in what location [MPS, Milwaukee (but not non-urban), non-urban]? • What is the candidates’ level of satisfaction with their field placements and supervision?

  15. 8) Job Location & Outlook Report • In what job locations do our completers work [MPS, Milwaukee (but not MPS), other large urban, non-urban]? • What is the employment outlook? [Using the DPI Supply and Demand Report; Table 20 – Supply ratings (Wisconsin vs. CESA #1); and Table 21 (Ratio of Applicants to Vacancies, Average Supply Rating, and Number of Emergency Licenses)]

  16. Report format, facilitating the use of data… • We try to: • keep the format straight forward – answering key questions (& avoiding collecting data that we will not use) • cluster programs-so we look beyond our own program • provide summary findings at the start of each report • make connections with other findings • include data over a 3-year period • note limitations

  17. II. Using the data

  18. Data Feedback Loop – Step 1 • Assessment & E-Portfolio Coordinator and the Data Management Team Collect Data and Develop Reports • Reports are addressed on a regular schedule: • Sept. Meeting - Exit Survey • Nov. Meeting - Standards & Portfolio; Follow-up • Feb. Meeting - Candidate Status; Praxis I & II • May Meeting - Job Location & Outlook

  19. Data Feedback Loop – Step 2 • CPE Meetings: Reports are Reviewed, Major Implications Discussed, Workgroups Convened • Steering Group reviews first • Draft report distributed in advance of CPE Meeting • There are benefits to the “public” discussion • “Unit view” taken –strengths noted; problem-solving approach to top items of concern • Workgroups convened as appropriate (e.g., Praxis Workgroup); Assessment tools & procedures revised as needed.

  20. Data Feedback Loop – Step 3 • Program Coordinators share results with program faculty and staff e Coordinators are asked to “bring back” & share findings with their program colleagues e Unit reports are made available online with a password e Individual program reports are sent directly to Coordinators and are not online

  21. Data Feedback Loop – Step 4 • Program Coordinators share answers to the following questions at the May Meeting and have opportunity to discuss: e What are the top 1-2 program strengths emerging from reports? e What are 1-2 areas of concern based on the data? e What actions are planned or underway & what progress has been made?

  22. Data Feedback Loop: Summary Points Using Data …influenced by: • The predictable flow of data (e.g., 8 annual reports on a regular schedule) • Data that matter– not cluttered with data that is of little consequence • The report itself (e.g., straight-forward; not a lot of narrative; key summary points up front)

  23. Data Feedback Loop: Summary Points Using Data …influenced by: • A process of engagement (e.g., data go to Steering Group first, then CPE, then to all programs & accessible online; in May – programs report out on how data used; and in 5-year review in Licensure Program Report) • A process that focuses on the “unit” & encourages support and concern for the whole

  24. Data Feedback Loop: Summary Points Using Data …influenced by: • Connections made among reports and other data • Workgroups that follow through on top concerns • Strengths are emphasized too (results might find their way into program documents, recruitment brochures, etc.)

  25. Data Feedback Loop: Summary Points Using Data …influenced by: • Climate – high concern for program quality, respect for work of program faculty, comfort with revealing and meaningfully addressing program weaknesses • An infrastructure that supports this work

  26. III. Your Turn • What else? What are your questions? • What strategies are working for you as you develop a system to ensure use of the data collected?

More Related