Outline

1. Outline • Definitions of Research/Evaluation • Purposes of Evaluation / Research • Science and Scientific Management • Research/Evaluation as Process • Types of Research / Evaluation

2. Definitions • Evaluation = Process of judging the merit or worth of something • Research • application of scientific methods to answer questions • controlled inquiry directed at increasing knowledge/establishing truth • Evaluation Research - combine the two

3. Research Evaluation Evaluation Research

4. Why Evaluate - the Academic list • To assess merits of alternative programs • To discover whether & how well objectives are being fulfilled • To determine the reasons for successes & failures. • To uncover the principles underlying a successful program. • To refine, revise, update or track a program

5. Why Evaluate- "The Real List" 1. Because we are required to 2. To make better decisions. 3. To learn from experience 4. To justify programs 5. To kill programs

6. Purposes of Research • Answer management questions - applied research • For sake of knowing - pure or basic research • Answer research questions - methodological • Develop/test management alternatives - developmental research • Assess worth or merit of programs - evaluationresearch

7. Research Purposes - II • Exploratory • Descriptive • Explanatory • Predictive

8. Ways of Knowing or Establishing belief or truth • Tradition • Authority • Repetition/Tenacity • Science

9. Body of Knowledge systematic abstract general parsimonious Method of Inquiry logical induction deduction self-corrective empirical Science

10. Scientific Management • Application of scientific principles to management and decision making • systematic information gathering • empirical, objective, self-corrective

11. Research define problem objectives/hypotheses literature review research methods gather data/analysis conclusions Evaluation describe program evaluation criteria program scoping evaluation methods gather data/analysis conclusions Process -- Steps

12. Types of Evaluationby Program Stage • formative (conceptualization/design)] • process (implementation) • summative (outcomes, impacts, efficiency)

13. Types - By Approach • Standards • norm-based • criterion-referenced • Goals and objectives • Impacts or effects

14. Types- by method • Qualitative - quantitative • Secondary data - Primary • Survey - Experiment • Internal vs external • Who - Peers, participants, superiors, subordinates, scientists

15. Evaluation Criteria • Effort - qnty and qlty of inputs • Performance - qnty and qlty of outputs • Adequacy - meet needs? • Efficiency - benefits/costs • Equity - distributional issues, fairness

16. Process Evaluation • Identifies how and why program works • attributes • recipients • conditions • effects • single or multiple • intended or side effects • timing & duration, long/short term • cognitive, affective or behavioral

17. Types by program area/subarea • Emphasis: Parks, Tourism, Interp, Program Mgmt, Commercial Rec, ... • Program : lands, facilities, personnel, budget, marketing, PR, maintenance, design, policies, plans

18. Types of Research • Basic - Applied • Exploratory, Descriptive, Explanatory, Predictive • In-house / out- house • by discipline - sociological, psych, economics • by topic - boating, wilderness, legal,tourism, ... • by method - survey, expmt, ...

19. Examples of Evaluation Studies • Community needs assessment • Feasibility study, SWOT analysis • Program Audits, Market audit • Cost/benefit anaysis • Impact assessment • PPBS, MBO, GPRA, CAPRA, LAC, ... • Peer review • Importance-performance analysis