1 / 31

Some Thoughts About Evaluating nsf’s investments in informal science education

Some Thoughts About Evaluating nsf’s investments in informal science education. Dr. Mark St. John Inverness Research Associates January 2005 www.inverness-research.org. INVERNESS RESEARCH. Inverness California. Inverness California. The Functions of Evaluation. Functions of Evaluation.

jacqui
Download Presentation

Some Thoughts About Evaluating nsf’s investments in informal science education

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Some Thoughts About Evaluating nsf’s investments in informal science education Dr. Mark St. John Inverness Research Associates January 2005 www.inverness-research.org

  2. INVERNESS RESEARCH

  3. Inverness California

  4. Inverness California

  5. The Functions of Evaluation

  6. Functions of Evaluation • Conceptualization, Documentation And Portrayal • Formative Feedback • Planning and Opportunity Analysis • Summative Evaluation • Research

  7. Functions of Evaluation

  8. EVALUATION AS A PROCESS OF GROUNDTRUTHING

  9. EVALUATING NSF INVESTMENTS NSF $$$ BENEFITS TO EDUCATION PROJECT OR ACTIVITY UNDERLYING “THEORY OF ACTION”

  10. Groundtruthing involves the comparison of theory of action (mental models) and field realities

  11. Evaluation as Groundtruthing: Comparing Project Logic and Field Realities Project Theory of Action (Assumptions, Intentions, Strategies Design Principles) Field Realities Evaluation helps to elucidate and refine Evaluation determines what is actually happening Evaluation Focuses On Congruence

  12. 1) Conceptualization, Documentation and Portrayal • Foundational function • Focuses on both theory of action and actual work done • Assists the project in basic documentation • Lays out the project in complete and analytical fashion so that all can see what it intends to be and what it is actually doing • Helps others understand the nature of the work of the project

  13. 2) Formative Feedback • “Critical friend” role; “Groundtruthing” • Working in relationship • Focus on issues, interactions, and information that will assist project in shaping design and implementation • Supports the co-evolution of project and evaluation design

  14. 3) Planning and opportunity analysis THE STRENGTHS AND ASSETS OF THE PROJECT IMPORTANT EDUCATIONAL NEEDS THE CURRENT LANDSCAPE OF SOCIETY (TRENDS:ECONOMICS, POLITICS, CULTURE)

  15. 3) Summative Evaluation • Helps others understand the theory, work and contributions of the project • Allow funders (and others) to assess return on investment and overall value of the effort • Places the project in broader contexts • Seeks creative methods to measure appropriate outcomes in rigorous ways • Can support the case for future funding

  16. SUMMATIVE: MAKING THE CASE (TYPES OF CLAIMS) CLAIMS THAT ARE TRUE CLAIMS THAT HAVE POLITICAL CURRENCY CLAIMS THAT ARE DOCUMENTABLE

  17. SUMMATIVE: MAKING A CASE DIFFERENT DIMENSIONS OF THE ARGUMENT AND DIFFERENT TYPES OF EVIDENCE

  18. 4) Research • Treats the project as a “case” • Helps the project summarize lessons learned • Studies the landscape the project works within • Studies the more general lessons learned vis-à-vis this type of investment • Helps the project generate knowledge as one of its contributions

  19. Suggested Steps in Project Evaluation • Prioritize (5) functions • Begin work with your evaluator around the theory of action • Use groundtruthing in a formative fashion • Decide on summative audience and their needs: • Make the case using: 1) project logic and 2) initiative goals or drivers • Decide if and how to use the evaluation to contribute to research effort

  20. Evaluating An investment in informal science education is evaluating a particular kind of NSF investment

  21. Different perspectives on informal science education

  22. I, Thou, IT –the relationship triangle The Watershed And its Phenomena (physical, biological, historical, cultural Economic) An Evironment and Institutional Context The Teacher, Guide, Adults Exhibit Designer The Vistor, Participant, Student…

  23. Organizational health Mission Personal and Collective Energy Structures, Relationships Resources

  24. Informal science learning:CULTURES AND SUBCULTURES MAINSTREAM CULTURE SCIENCE – INTERESTED SUBCULTURE ENVIRONMENT – INTERESTED SUBCULTURE

  25. THE SPECTRUM OF ACCULTURATION DIS-INTERESTED HIGHLY INTERESTED INFORMAL EXPERIENCES FORMAL EXPERIENCES

  26. AUDIENCES AND TEACHERS PROFESSIONALS ADVANCED AMATEURS ATTENTIVE PUBLIC READY PUBLIC MAINSTREAM PUBLIC

  27. NSF’s Relationship to the Informal Learner (NSF To Learner – 3 Degrees Of Separation)

  28. The First Degree SCIENCE LEARNING AND ACCULTURATION INFORMAL LEARNING OPPORTUNITIES SEMI-FORMAL LEARNING OPPORTUNITIES FORMAL LEARNING OPPORTUNITIES

  29. The SECOND Degree SCIENCE LEARNING AND ACCULTURATION INFORMAL LEARNING OPPORTUNITIES FORMAL LEARNING OPPORTUNITIES SEMI-FORMAL LEARNING OPPORTUNITIES INSTITUTIONS AND SYSTEMS (Formal, Semi-Formal, Informal)

  30. The THIRD Degree PUBLIC LEARNING AND ACCULTURATION INFORMAL LEARNING OPPORTUNITIES FORMAL LEARNING OPPORTUNITIES SEMI-FORMAL LEARNING OPPORTUNITIES INSTITUTIONS AND SYSTEMS (Formal, Semi-Formal, Informal) PROJECTS ADD VALUE NSF $$$

  31. Simultaneously funding the capacity and the work of the project Develop capacity to do the work INVESTMENT $$$ Project activities to do the work

More Related