Measuring
Download
1 / 18

Measuring the Ephemeral: Evaluation of Informal STEM Learning Experiences Broader Impacts Infrastructure Summit Arl - PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on

Measuring the Ephemeral: Evaluation of Informal STEM Learning Experiences Broader Impacts Infrastructure Summit Arlington, VA April 16 th -18 th , 2014. Images courtesy of ISE PI Meeting 2012 attendees

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Measuring the Ephemeral: Evaluation of Informal STEM Learning Experiences Broader Impacts Infrastructure Summit Arl' - dmitri


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Measuring the Ephemeral:

Evaluation of Informal STEM Learning Experiences

Broader Impacts Infrastructure Summit

Arlington, VA

April 16th-18th, 2014

Images courtesy of ISE PI Meeting 2012 attendees

From left to right: Geoffrey Haines-Stiles; Mohini Patel Glanz, NWABR; Scot Osterweil; April Luehmann


Jamie Bell, Trevor Nesbit, Kalie Sacco, Grace Troxel (Association of Science-Technology Centers)

John Falk (Oregon State University, Free-Choice Learning Program)

Kirsten Ellenbogen (Great Lakes Science Center)

Kevin Crowley (University of Pittsburgh Center for Learning in Out-of-School Environments)

Sue Ellen McCann (KQED Public Media)


Museums & science centers (Association

Zoos Botanical Aquaria

Media (TV, Radio, Film)

NSF AISL Program

Festivals, cafes, events

Cyber & Gaming

Youth & Community Programs

Citizen Science

caise

Convene, Connect, Characterize, Communicate

Broader Impacts/ISE

informal

science.org

Practice-and- Research

Evaluation Capacity Building

^CAISE major initiatives for 2012-2015^



Learning Science in Informal Environments: People, Places and Pursuits

National Academies Press, 2009

Surrounded by Science: Learning Science in Informal Environments

National Academies Press, 2010


Learning Science in Informal Environments and Pursuits(2009)

Six Learning Strands:

Developing interest

Understanding scientific knowledge

Engaging in scientific reasoning

Reflecting on science

Engaging in scientific practices

Identifying with the scientific enterprise


Taking Science to School: and Pursuits

Learning and Teaching Science in Grades K-8 (2007)

K-8 School science learning strands:

Blank

Understanding scientific explanations

Generating scientific evidence, explanations and arguments

Reflecting on how science knowledge is produced and used in society

Participating in the practices of science—specialized talk, disciplinary tool use, representations


Informal learning and Pursuitsenvironments are complex.

  • Challenges:

  • Cannot separate single experience

  • Experimental design is impractical

  • Opportunities:

  • Allow wide range of outcomes

  • Naturally learner-driven

  • Inspires new methods and approaches

Many different types of learning experiences and interactions can take place within a single exhibition.

Image attribution:

How People Make Things Summative Evaluation

Camellia Sanford, University of Pittsburgh, 2009

Accessed March 20th, 2014:

http://informalscience.org/evaluation/ic-000-000-003-205/


Informal learning is and Pursuitscollaborative and social.

  • Challenges:

  • Individual assessment can be difficult

  • Opportunities:

  • Helps us better understand how people learn

High school students collaborate with an ecologist to study aquatic ecosystems.

Image attribution:

Making Natural Connections: An Authentic Field Research Collaboration

Accessed March 20th, 2014:

http://informalscience.org/projects/ic-000-000-001-050


Liftoff evaluation framework
LIFTOFF Evaluation Framework and Pursuits

Data Collection: # of programs offering STEM and the intensity of those experiences

Youth Impacts: attitudes, motivation, identity

Professional Impacts: attitudes, demand, confidence

Program Quality Impacts: frequency and intensity


12 dimensions of success
12 Dimensions of Success and Pursuits

www.PEARweb.org


Resources for Evaluation: and Pursuits

The PI’s Guide to Managing Evaluation

InformalScience.org/evaluation/evaluation-resources/pi-guide


Resources for Evaluation: and Pursuits

Evaluations on InformalScience.org

InformalScience.org/evaluation


Building Informal Science Education (BISE) Project: and Pursuits

Characterizing evaluations on InformalScience.org

300

275

275

261

250

225

200

Frequency of data collection methods used in the BISE synthesis reports (n=427)

175

150

128

125

100

73

75

69

50

40

25

14

11

10

10

9

8

8

6

5

3

5

6

1

0

Observation

Survey

Interview

Drawings

Other

Professional critique

Timing and tracking

Participation data

Card sort

Focus group

Artifact review

Journals

Web analytics

Concept map

Delphi technique

Interactive methods

Didn’t describe method

Recorded conversations

Comment cards and books


Informal STEM Education Assessment Projects and Pursuits

  • Advancing Technology Fluency (PI: Brigid Barron, Stanford University)

  • Developing, Validating, and Implementing Situated Evaluation Instruments (DEVISE) (PI: Rick Bonney, Cornell University)

  • Common Instrument (PI: Gil Noam,

  • Harvard University)

  • Framework for Observing and Categorizing Instructional Strategies (FOCIS) (PI: Robert Tai, University of Virginia)

  • Science Learning Activation Lab (PI: Rena Dorph, University of California, Berkeley)

  • SYNERGIES (PI: John Falk, Oregon State University

Learn more at:

http://informalscience.org/perspectives/blog/updates-from-the-field-meeting-on-assessment-in-informal-science-education


What are We and PursuitsMeasuring?

This Wordle represents constructs measured by six informal STEM education assessment projects.


Learn more… and Pursuits

Jamie Bell, Project Director & PI of CAISE

[email protected]

InformalScience.org

facebook.com/informalscience

@informalscience


InformalScience.org/about/informal-science-education/for-scientistsInformalScience.org/about/informal-science-education/for-scientists


ad