1 / 25

Evaluation

I N T E R N A T I O N A L E D U C A T I O N P R O G R A M S S E R V I C E. Evaluation . CIBE, FLAS, IIPP, LRC, NRC Technical Assistance Workshop February 2-3, 2009. Access to the World and Its Languages www.ed.gov/ope/iegps. Title VI 50 th Anniversary Update.

stan
Download Presentation

Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. I N T E R N A T I O N A L E D U C A T I O N P R O G R A M S S E R V I C E Evaluation CIBE, FLAS, IIPP, LRC, NRC Technical Assistance Workshop February 2-3, 2009 Access to the World and Its Languageswww.ed.gov/ope/iegps

  2. Title VI 50th Anniversary Update • Madeline Albright is the Keynote speaker • Please make separate reservations • Conference registration • Hotel registration • Rooms are going quickly • Contacts Rob Glew and Tomas Hult, Michigan State University • http://titlevi50th.msu.edu

  3. Love Evaluation

  4. Panelists • Eleanor Briscoe, Senior Budget Analyst • John Clement, Director, Strategic Planning Services • Susan Lehmann, Evaluation Analyst, FIPSE • Karla Ver Bryck Block, Senior Advisor, IEPS

  5. Why??? • ED believes in and supports high quality program evaluations to help inform budgetary and program decisions • ED Strategic Plan • Goal 3, Objective 1: Increase success in and completion of quality postsecondary education. • Strategy 3. Prepare more graduates for employment in areas of vital interest to the United States, especially in critical-need languages, mathematics, and the sciences.

  6. Administration Update • “Accountability and Transparency” • GPRA continues • PART may or may not continue but the need to measure program effectiveness remains

  7. Congress • Clear interest in program evaluations • Directed a $1.5 million review of Title VI and Fulbright-Hays programs by the National Research Council of the National Academies • International Education and Foreign Language – Keys to Securing America’s Future (March 2007)

  8. Keys to Securing America’s Future • National Academies recommendation • The Department of Education should commission independent outcome and impact evaluations of all programs every 4 to 5 years

  9. IEPS Program Level Initiatives • Evaluations are underway with AIR • Language Resource Centers • Fulbright-Hays Group Projects Abroad – Overseas Intensive Language Projects • Analysis of the International Research and Studies Program is concluding • National Resource Centers are next • Future plans include Centers for International Business Education

  10. IRIS • IEPS gathers information via IRIS that is used for program evaluations • Breakouts for 4 programs today (CIBE, LRC) and tomorrow (NRC, FLAS) to discuss: • Current and future program performance and efficiency measures • Changes to IRIS for the next clearance • Evaluation efforts • Trivia, prizes, chocolate!!!

  11. Keys to Securing America’s Future • National Academies Recommendation • The Department of Education should work with universities to create a system of continuous improvement for Title VI and Fulbright-Hays programs. The system would help develop performance indicators and other improvement tools and should include networks of similar centers and university officials with overall responsibilities in language, area, and international studies.

  12. Your Project • Think… • Goal-oriented • Data driven ED does not have specific models, timelines, budgets, etc for your evaluation plans. Develop a plan that serves your needs.

  13. Performance Objectives & Measures • A measurable indication used to determine how well goals are being met • How will progress be measured? • How much progress will constitute success? • How will it be known if an objective or part of an objective has been achieved? • Are we measuring process or outcome?

  14. Objective & Measure Types • A specific achievement/accomplishment • Change in a level of achievement/accomplishment • Achievement/accomplishment (or change) compared with other students or institutions

  15. Components of Objectives & Measures • What will change • Howmuch change is expected • Who will achieve the change • When the change will take place

  16. Performance Objective Example • Help local businesses to internationalize “20% (how much) of local businesses with more than 200 employees (who) will open an office in a foreign country or enter into a joint venture with a foreign company (what)within two years (when).”

  17. Performance Objective Example • Teacher workshop – Developing Materials for LCTLs “50 (how much) LCTL teaching professionals (who) will complete LCTL workshop (what)annually (when).” “90% (how much) of LCTL workshop participants (who) respond with 4 or higher on survey (what)one year after completing program (when).”

  18. Performance Measure Example • Revise public health curriculum to enhance international knowledge “The required MPH courses (who) will be revised (what)to cover the 10 important international health areas (how much) by next fall (when).”

  19. Performance Measure Example • Improve enrollment in summer language institute “Undergraduate student enrollment in the summer language institute (who) will increase (what)by five percent (how much) next summer (when).”

  20. Improving Performance Measures • Original Performance Measure: Provide overseas internships. • What/Who: • Provide dynamic student internships with partners in a foreign country. • When: • Provide dynamic student internships with partners in a foreign country each semester. • How Much: • 20 students will engage in dynamic student internships with partners in a foreign country each semester.

  21. Improving Performance Measures • Original Performance Measure: Enhance Business Resource Center. • What/Who: • Enhance Business Resource Center to advise businesses on international trade regulations. • When/How Much: • Business Resource Center documents and staff will be updated on new international trade regulations within one week of passage.

  22. Remember!!! • Activities are not performance measures!

  23. Resolving the “Activity-as-Measures” Dilemma • “Create opportunities for local faculty to strengthen area skills.” (activity) • Create “visiting” faculty training programs and train 2 per year. (outcome) • “Enhance study abroad.” (activity) • Increase number of students studying abroad by X% each year. (outcome)

  24. Summary: Good Objectives & Measures • Each objective should have one or a few specific performance measures to demonstrate how progress will be measured toward meeting the objective • More objectives and measures ≠ better objectives and measures! • Ensure data is collectible • Quantify and define success

  25. Summary: Strong Evaluation • Partner with a competent evaluator • Develop quantifiable objectives and measures • Measure what is important • Measure what can be changed • Get frequent progress reports • Keep key stakeholders informed of results • Use results to adjust project regularly • Keep analyzing…

More Related