Measuring
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

Measuring the Ephemeral: Evaluation of Informal STEM Learning Experiences Broader Impacts Infrastructure Summit Arlington, VA April 16 th -18 th , 2014 PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on
  • Presentation posted in: General

Measuring the Ephemeral: Evaluation of Informal STEM Learning Experiences Broader Impacts Infrastructure Summit Arlington, VA April 16 th -18 th , 2014. Images courtesy of ISE PI Meeting 2012 attendees

Download Presentation

Measuring the Ephemeral: Evaluation of Informal STEM Learning Experiences Broader Impacts Infrastructure Summit Arlington, VA April 16 th -18 th , 2014

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Measuring the Ephemeral:

Evaluation of Informal STEM Learning Experiences

Broader Impacts Infrastructure Summit

Arlington, VA

April 16th-18th, 2014

Images courtesy of ISE PI Meeting 2012 attendees

From left to right: Geoffrey Haines-Stiles; Mohini Patel Glanz, NWABR; Scot Osterweil; April Luehmann


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Jamie Bell, Trevor Nesbit, Kalie Sacco, Grace Troxel (Association of Science-Technology Centers)

John Falk (Oregon State University, Free-Choice Learning Program)

Kirsten Ellenbogen (Great Lakes Science Center)

Kevin Crowley (University of Pittsburgh Center for Learning in Out-of-School Environments)

Sue Ellen McCann (KQED Public Media)


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Museums & science centers

Zoos Botanical Aquaria

Media (TV, Radio, Film)

NSF AISL Program

Festivals, cafes, events

Cyber & Gaming

Youth & Community Programs

Citizen Science

caise

Convene, Connect, Characterize, Communicate

Broader Impacts/ISE

informal

science.org

Practice-and- Research

Evaluation Capacity Building

^CAISE major initiatives for 2012-2015^


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Framework for Evaluating Impacts of Informal Science Projects (2008)


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Learning Science in Informal Environments: People, Places and Pursuits

National Academies Press, 2009

Surrounded by Science: Learning Science in Informal Environments

National Academies Press, 2010


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Learning Science in Informal Environments (2009)

Six Learning Strands:

Developing interest

Understanding scientific knowledge

Engaging in scientific reasoning

Reflecting on science

Engaging in scientific practices

Identifying with the scientific enterprise


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Taking Science to School:

Learning and Teaching Science in Grades K-8 (2007)

K-8 School science learning strands:

Blank

Understanding scientific explanations

Generating scientific evidence, explanations and arguments

Reflecting on how science knowledge is produced and used in society

Participating in the practices of science—specialized talk, disciplinary tool use, representations


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Informal learning environments are complex.

  • Challenges:

  • Cannot separate single experience

  • Experimental design is impractical

  • Opportunities:

  • Allow wide range of outcomes

  • Naturally learner-driven

  • Inspires new methods and approaches

Many different types of learning experiences and interactions can take place within a single exhibition.

Image attribution:

How People Make Things Summative Evaluation

Camellia Sanford, University of Pittsburgh, 2009

Accessed March 20th, 2014:

http://informalscience.org/evaluation/ic-000-000-003-205/


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Informal learning is collaborative and social.

  • Challenges:

  • Individual assessment can be difficult

  • Opportunities:

  • Helps us better understand how people learn

High school students collaborate with an ecologist to study aquatic ecosystems.

Image attribution:

Making Natural Connections: An Authentic Field Research Collaboration

Accessed March 20th, 2014:

http://informalscience.org/projects/ic-000-000-001-050


Liftoff evaluation framework

LIFTOFF Evaluation Framework

Data Collection: # of programs offering STEM and the intensity of those experiences

Youth Impacts: attitudes, motivation, identity

Professional Impacts: attitudes, demand, confidence

Program Quality Impacts: frequency and intensity


12 dimensions of success

12 Dimensions of Success

www.PEARweb.org


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Resources for Evaluation:

The PI’s Guide to Managing Evaluation

InformalScience.org/evaluation/evaluation-resources/pi-guide


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Resources for Evaluation:

Evaluations on InformalScience.org

InformalScience.org/evaluation


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Building Informal Science Education (BISE) Project:

Characterizing evaluations on InformalScience.org

300

275

275

261

250

225

200

Frequency of data collection methods used in the BISE synthesis reports (n=427)

175

150

128

125

100

73

75

69

50

40

25

14

11

10

10

9

8

8

6

5

3

5

6

1

0

Observation

Survey

Interview

Drawings

Other

Professional critique

Timing and tracking

Participation data

Card sort

Focus group

Artifact review

Journals

Web analytics

Concept map

Delphi technique

Interactive methods

Didn’t describe method

Recorded conversations

Comment cards and books


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Informal STEM Education Assessment Projects

  • Advancing Technology Fluency (PI: Brigid Barron, Stanford University)

  • Developing, Validating, and Implementing Situated Evaluation Instruments (DEVISE) (PI: Rick Bonney, Cornell University)

  • Common Instrument (PI: Gil Noam,

  • Harvard University)

  • Framework for Observing and Categorizing Instructional Strategies (FOCIS) (PI: Robert Tai, University of Virginia)

  • Science Learning Activation Lab (PI: Rena Dorph, University of California, Berkeley)

  • SYNERGIES (PI: John Falk, Oregon State University

Learn more at:

http://informalscience.org/perspectives/blog/updates-from-the-field-meeting-on-assessment-in-informal-science-education


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

What are We Measuring?

This Wordle represents constructs measured by six informal STEM education assessment projects.


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

Learn more…

Jamie Bell, Project Director & PI of CAISE

[email protected]

InformalScience.org

facebook.com/informalscience

@informalscience


Measuring the ephemeral evaluation of informal stem learning experiences broader impacts infrastructure summit arlington va april 16 th 18 th 2014

InformalScience.org/about/informal-science-education/for-scientists


  • Login