Assessing affective outcomes in ADVENTURE PROGRAMMING

1. Assessing affective outcomes in ADVENTURE PROGRAMMING Matthew Cummiskey John G. Helion West Chester University of PA

2. Q&A • Why is assessing these important in both school and community based adventure programs? • What are you doing presently to assess?

3. Rationale • PE National Standards • Standard 5: Exhibits responsible personal and social behavior that respects self and others in physical activity settings. • Standard 6: Values physical activity for health, enjoyment, challenge, self-expression, and/or social interaction. • “Titanic Thinking”

4. Identifying the Soft Skills • What are some “soft skills” that can be assessed? • Teamwork • Cooperation • Communication • Problem Solving • Conflict Resolution • Responsibility • Trust • Others

5. Types of Assessments

6. Checklists • Use Checklists When: • There is only one correct method, little or no shades of gray. • Usually more simplistic items being measured (brake hand never leaves rope)

7. Rating Scales • Used when the product can be broken into discreet process parts that affect product • Qualitative, • Always, Sometimes, Never • Rating between scorers can be inconsistent/unreliable • Example – correctly reading compass bearings • Quantitative • 4 times, 3 times etc. • Example – correctly tie a retraced figure 8 all the time v. one error v. two errors etc.

8. Analytic Rubrics • A rubric is a scale of criteria that indicates and identifies levels of performance • belay technique • What most people think of • Difficult to create but once in “can” yours forever • Improved validity/reliability issues when used to grade

9. Analytic Rubric Basics • Shared with Ss as the unit or project is explained • Encourage Ss to read prior to the assessment so they know how they will be evaluated • There are millions of rubrics on the internet • Easier to find one and modify it than write your own • Different aspects of a rubric can be “weighted” to reflect importance of the various

10. Reliability of Analytic Rubrics • Reliability: Will the assessment measure learning consistently regardless of who is doing the assessing? • Avoid use of “qualifier” words

11. Steps in writing a rubric 1. Envision the desired student performance • Setting up an adventure activity (spider web, wild woozy etc.) 2. Determine the number of scoring columns • Proficient, developing, basic & below basic • 5, 4, 3 or 2 – why is the # of columns important? • Assign “weights” to each row (criteria) 3. Complete the cells • Describe performance in detail, focus on quality and not quantity • Limit # of threads to 2 or 3 per row 4. Use with students and revise • Multi-trait rubric • Character development rubric (elem PE)

12. Selecting Rubric Items/Criteria • If you were developing a rubric for 12th graders or a community group climbing on the rock wall, what items would you select? (aka...what goes in the left column) • Teamwork • Group Focus • Support • Perseverance • Prep Work

13. Group Work • Develop a rubric for climbing a rock wall • Example

14. Scoring • Avoid having a rubric printed for every participant, instead develop a scoring grid for everyone and refer to one printed rubric. • Data table for rubric shown earlier • Assume everyone gets full credit unless indicated • Vary the number of skills assessed based upon the activity, number of students and complexity of the assessment • The data table above would take multiple days to complete

15. Finding Assessments • Computer Searches • Google.com • Ask.com • Yahoo.com • Descriptors • General • Rubrics – Thousands of hits • Specific • Teamwork Rubrics • Cooperation Rubrics • Be selective – There are many options

16. Individual Affective Rubrics • Level of involvement 1, 2 • Oral communication • When developing individual affective rubrics, often best to consider its definition and work backwards from there.

17. Teamwork • Joint action by a group of people, in which individual interests are subordinated to group unity and efficiency; coordinated effort • From teacher glossary

18. Cooperation • Co-operation refers to the practice of people or greater entities working in common with commonly agreed-upon goals and possibly methods, instead of working separately in competition. en.wikipedia.org/wiki/Cooperation

19. Communication • The transmission of information so that the recipient understands what the sender intends.www.welcom.com/content.cfm • The successful transmission of information through a common system of symbols, signs, behavior, speech, writing, or signals.www.doe.mass.edu/frameworks/scitech/2001/resources/glossary.html

20. Problem Solving • A systematic approach utilizing multiple perspectives to uncover the issues related to a particular problem, design an intervention plan, and evaluate the outcome.rrtcpbs.fmhi.usf.edu/rrtcpbsweb/glossary.htm • The act of defining a problem; determining the cause of the problem; identifying, prioritizing and selecting alternatives for a solution; and implementing a solution.www.onesixsigma.com/tools_resources/glossary/glossary_p.php

21. Conflict Resolution • Conflict resolution is the process of resolving a dispute or a conflict, by providing each side's needs, and adequately addressing their interests so that they are satisfied with the outcome. Conflict resolution aims to end conflicts before they start or lead to physical fighting. en.wikipedia.org/wiki/Conflict_resolution

22. Trust • Firm reliance on the integrity, ability, or character of a person or thing. Custody; care.www.answers.com/topic/trust

23. Qualitative Analysis • Ask students an open-ended question that requires a written response. For example, “What did you learn today that helped you grow as a person.” Or use several open-ended question relating to different topics: “How did you improve your teamwork from today’s activities.” “Describe what may have changed about the group as a result of today’s activities.”

24. Qualitative • Group all of the responses from each open-ended questions together but separate by question. Next read all the responses and look for themes (ideas that reoccur), write down the themes separately. You may have subthemes under the major themes. Next, re-read all the responses and tally each time a theme (or subtheme if you have them) is present. You will begin to notice patterns in the data which can be represented numerically. For example, you may find that “building rapport” is the most common theme mentioned 126 times, next is “communicating ones ideas” mentioned 89 times and so. Create an exhaustive list (you may have 10, 20, 30 themes) along with a tally number for each. Finally, discuss what the data means and use quotations to underscore the points you are making. For more information look online, this method is called the “constant comparison method.”

25. Parting Thoughts • Time is a commodity • Assessment doesn’t have to be cumbersome • Put in the time to improve your instruction/program • Millions of words are put in the internet every day, don’t need to reinvented the wheel • Start small and ramp up over time

26. E-mail Address • Matthew Cummiskey • mcummiskey@wcupa.edu • John Helion • jhelion@wcupa.edu • Materials available at: http://thenewPE.com (click conference presentations link) Thanks for coming. Enjoy the rest of the conference!