1 / 20

In-depth Evaluation of R&D Program in Korea

This report provides an overview of the public R&D program evaluation in Korea, including the objective, procedure, methodologies, utilization of evaluation results, and challenges. It also highlights the importance of efficient R&D management and the need for stakeholder understanding, change, and improvement. Contact Dr. Seung Jun Yoo at biojun@kistep.re.kr for more information.

Download Presentation

In-depth Evaluation of R&D Program in Korea

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. In-depth Evaluation of R&D Program in Korea Seung Jun Yoo, Boo-jongGill, Woo ChulChai

  2. Contents 1. Overview of Public R&D Program Evaluation 2. Objective 3. Procedure 4. Methodologies 5. Utilization of Evaluation Results 6. Challenges and Discussion

  3. Agency Agency Agency Agency Agency Overview of Public R&D Program Evaluation 1 - Players Report Taking Charge of R&D Evaluation and Budget Allocation MOSF NSTC Evaluation Supporting Groups KISTEP (Evaluator) R&D programs of each ministry MIFAFF MOE MKE MW MEST …… *NSTC (National Science & Technology Council), MOSF(Ministry of Strategy and Finance)

  4. Evaluation strategy & Data collection R&D Budget Survey/Analysis Evaluation Programs/Projects implemented In-depth Corrections* Feedback Overview of Public R&D Program Evaluation 2 - R&D program management process Self → Meta Evaluation process Recommendations to program ministries Correction process

  5. Overview of Public R&D Program Evaluation 3 - Overview of Public Finance Program

  6. Overview of Public R&D Program Evaluation 4 - In-depth evaluation & Self/Meta evaluation In-depth Evaluation • ~ 10 programs with evaluation issues • logic model, evaluation questions, in- • depth analysis, communications, • recommendations, coordination, etc. • - depends on evaluation questions R&D Program Evaluation Self → Meta Evaluation • 1/3 out of all programs • (70 programs out of 207 programs, ‘09) • self evaluation by each ministry • meta evaluation by MOSF/KISTEP • - depend on indicators with weight

  7. Objective of the Evaluation • To increase the efficiency and effectiveness • : find out and diagnose the problems at all aspects • : improve the program by applying evaluation results

  8. Procedure 1 - 4(5) steps 0. Selecting target program

  9. Procedure 2 - 7-month schedule (depends on the cases) - (month 0) : selected by selection committee based on special issues, etc. ¶ In-depth evaluation procedure for selected program(s) - month 1 : form evaluation group, gather program(s) data, study target R&D program(s), find major evaluation questions - month 2 : develop logic model and methodologies - month 3/4 : perform in-depth analysis (relevance, efficiency, effectiveness, program design & delivery, etc)

  10. Procedure 3 - month 5 : interview (researchers, program managers, etc.) - month 5 : report interim evaluation result (MOSF, ministries) - month 6 : report final evaluation result & recommendations Large program : ~ 10 months Specific needs for short-term : 2 ~ 3 months (specify the needs → perform evaluation)

  11. Methodologies

  12. Methodologies_IPA

  13. Methodologies_Survival Analysis Following Study Modified from: DeVol, R. & Bedroussian, A.(2006), Mind to Market: A Global Analysis of University Biotechnology Transfer and Commercialization. Milken Institute

  14. Methodologies_System Dynamics Source: Ahn, N. (1999), A system dynamics model of a large R&D program, MIT Press Yoo, S. et al. (2009), In-depth Evaluation of Health & Medical R&D Program in Korea, KISTEP

  15. Methodologies_PortfolioAnalysis

  16. Utilization of Evaluation Results

  17. Self/Meta Challenges and Discussion Efficient R&D Management In-depth • Evaluation as R&D Management Tool Measure the achievement according to performance plan • Efficient budgeting • Improve goal achievement Diagnose problems and correct to improve the efficiency/effectiveness

  18. Challenges and Discussion • UCI concept among stakeholders • : Understanding – Change - Improvement • is important for raising the accountability • (responsibility + acceptability) of evaluation • : Understanding = communication with the facts • : Change = 4 types of corrections • : Improvement = efficiency & effectiveness

  19. Thank you! Seung Jun Yoo, PhD biojun@kistep.re.kr www.kistep.re.kr

More Related