Download
1 / 63
630 likes | 756 Views
E N D
1. Developing Mountain Navigation Skills in Outdoor Education: An Evaluation of Questionmark Perception, Self and Peer Assessment Tim Stott, Andy Boorman, Dave Hardy
Centre for Outdoor & Environmental Education
School of The Outdoors, Leisure & Food, Faculty of Education, Community & Leisure
2. Background The Mountain Leadership Award (MLA) is integrated into the BSc (Hons) Outdoor and Environmental Education programme principally because it has become the gold standard in mountain leadership in the UK, sometimes being a legal requirement, and it provides a valuable and well recognised syllabus
The MLA scheme is a comprehensive professional award that requires a significant commitment on the part of students if they are to complete it
Training and assessment for this award has been delivered as part of Outdoor Education degree programmes at IM Marsh for over 30 years.
4. developing students mountain navigation skills in OLFEO1201: Outdoor Pursuits Leadership 1, a Level 1 module is seen by staff, students and employers as a central core skill.
The ability to navigate in hostile mountainous terrain, often in bad weather, poor visibility, at night or a combination of these, is a demanding skill which teachers and leaders in Outdoor Education are required to demonstrate
6. 1. to design, use and evaluate a computer based learning tool for teaching and assessing the theoretical aspects of mountain navigation based on Questionmark Perception
2. to develop and evaluate the use of self and peer-assessment techniques to assist students to identify strengths and weaknesses in their practical navigation performance in the mountains (LJMU Year 1 OEE students on Module OLFEO1201) AIMS
8. Teaching and Learning context in common with many areas of higher education, students are expected to plan and action their own learning activities in order to reach the required standard for assessment
this approach is encouraged by the MLA structure outlined already
learning mountain navigation presents special challenges in that students practical skills in outdoor situations must often be developed independently in potentially hazardous situations.
9. ETHICAL CONSIDERATIONS This project is designed to raise participant awareness of leadership issues by assisting students to evaluate and develop their navigation skills.
Participation was voluntary.
Participants were kept fully informed of the project design and activities.
Confidentiality was respected throughout
Observations made by fellow participants were not used in any formal assessment process by the supervising lecturers
10. Aim 1: Computer Assisted Assessment thorough preparation of students to engage safely in this vital independent learning is particularly important.
use of Questionmark Perception (QP) is a vital strand in this preparation in ensuring that practical performance is preceded by a sound understanding of underpinning theoretical principles.
allows students to make errors, check their own accuracy and knowledge and work at their own pace until they are satisfied with their performance
11. Questionmark Perception (QP) advertises itself as the future of assessment testing
made available as an institutional assessment survey tool on the JMU Network early in 2003 see Alice Birds article in Volume 3, Issue 1 (Summer 2003) of JMU Learning & Teaching Press
13. QP v Blackboard (v5.5) Assessment Manager more sophisticated range of question types
Eg. Drag and Drop (see Bird, 2003, Fig. 1 on p.18) and Numeric question types were deemed to be useful additional features
14. As a learning experience for myself, and as a diagnostic and self-assessment tool for students, I developed a Mountain Navigation Test using QP (available at http://wsl.livjm.ac.uk/q/open.dll ).
18. The test consisted of 22 questions that included 17 images (14 extracts of 1:25 000 and 1: 50 000 Ordnance Survey maps and three other diagrams
question types used included nine Multiple Choice, four Drag and Drop, four Numeric, four Matching and one Fill in Blanks type.
25. Test designed to assess: students knowledge of map references,
estimating distances on different scale maps
use of map scales,
familiarity with compasses and direction,
contours (shapes, intervals and height differences),
estimating walking times, and
various navigation techniques.
option for questions appear in a random order was selected so that students sitting next to each other at computer terminals were less likely to collaborate and more likely to work independently.
27. Results 23 of the 32 students on the module took up the invitation to sit the QP Mountain Navigation Test
17 completed a paper based evaluation immediately afterwards and before leaving the IT room
33. Comments Specific to this Test Clear; precise; informative; can I do all my exams this way please ?
Some questions required too much scrolling up and down the page to find the correct answers (5); some questions did not show the correct answer in the feedback at the end; arrows on compass question not accurate enough; too much calculation; found it difficult to work with map scales and estimate distances on screen.
34. Experience of designing and publishing the test as a first-time QP user: Tim Stott registered interest on the relevant CIS page http://cwis.livjm.ac.uk/cis/perception/welcome.htm
Question Manager: prepare questions
Assessment Manager: publish the test on JMU Server (name image files with Username first)
Enterprise Manager (interrogate students results)
36. Aim 2: Self and Peer Assessment a structured self and peer assessment exercise designed to help students identify for themselves their strengths and weaknesses in practical mountain navigation
designed to take students from the use of rationalised technology described so far, to independent learning in context
37. The Practical Teaching Context: Planning and Conceptualisation interpretation of the map as a guide to the terrain
an understanding of the weather and underfoot conditions
an understanding of the influence of both on individuals within the group being led
38. examines the way that staff encourage students to develop judgement and assess their own decisions
approach is mainly via a task oriented experiential learning model
Reflection and reviewing, re-enforced by staff judgement, is used to encourage learning, whilst staff intervention is withdrawn to encourage student independence,
peer assessment exercise being the last stage along the tutor supported path to independence
39. Staged Experiential Learning: From reviewing to independence Stage 1 relies partly on reviewing the learning process following the agenda set by the instructor (Dewey, 1938; Kolb, 1984)
Stage 2 initiates a trend away from a hierarchical teaching/learning approach towards independent learning (Fenwick, 2001; Mosston and Ashworth, 2002)
40. At Stage 3 students are encouraged to reflect in action (Schön, 1987).
In Stage 4 the learner now moves closer to the heart of the learning process by defining his or her own agenda (the four Ds Model is employed as a starting point)
Stage 5 focuses on pure experience, not supplemented by reviewing, it has no structure other than that decided on by the learner
42. Problem 1: The four Ds model The intention of this was to help students to see each navigation technique in context and give them a tool for deciding independently what is the most appropriate set of techniques to apply to a particular navigation task (see Fig. 1 next slide)
44. model arrived at after surveying the basic strategies used by staff at JMU to teach mountain navigation
model is a over-simplistic, navigation tasks are rarely solved by a simple linear solution,
it is a useful starting point in developing a navigational strategy
Students encouraged to use the model in order to decide on which level of precision and speed they wish to develop
45. Problem 2: The Peer Assessment Exercise During previous evening students working in groups of three planned a route with a series of navigation legs
JMU tutors encouraged students to match the difficulty of the legs to their experience using the 4Ds model
46. Each group of three students consisted of one navigator (leader) whose skills were assessed by his/her two peers.
The leader rotated so that each member of the group led 3-4 navigation legs throughout the half-day exercise.
The navigational skills of the leader were assessed according to the scheme identified in Fig. 2
48. The peer assessment exercise was completed when students transferred their field notes from waterproof notebooks onto copies of Fig. 2, a laminated copy of which was given to each group of 3 for reference in the field.
49. Evaluation Completion of a brief individual evaluation questionnaire (Fig. 3).
Open discussion with trainers, details being tape-recorded and later summarised.
Open discussion with two students that was recorded and transcribed.
51. Student Reaction to the Peer Assessment Exercise and the use of the 4Ds Model
53. There is clear agreement (21 out of 24 responses or 88%) among the sample that suggests the exercise was very useful or extremely useful to their learning and the Chi-Square Test confirms (p < 0.001) that there is a very low probability that this outcome might have occurred by chance.
54. Question 2 Would you find it a useful model to use as part of your further development? 21 said yes and 3 said no. Two of these negative replies came from very inexperienced students, an initial indication that this navigation exercise is probably more suited to candidates with some previous expertise.
55. Qualitative/descriptive data from: students written comments;
the taped and summarised group discussion;
the recorded and transcribed discussion with two of the students
56. The 4Ds system as a learning tool and a model for further personal development. Model provides a safe but challenging framework for independent learning (10)
Very helpful on improving skills and using as a guide to performance (9)
Useful to revise/reinforce knowledge and identify areas of weakness (6)
A very enjoyable learning experience (5)
Gives the navigator more confidence (5)
Useful model for teaching others (3)
57. The value of the peer assessment/feedback system. I really enjoyed and appreciated the input of group members this helped me to understand what I needed to work on and improve (7)
Really made you focus and concentrate (3)
Motivated me to try harder (3)
Opportunities to evaluate different ways of navigation used by other group members (2)
Useful tools for feedback and assessing performance (2)
Has not told me anything I did not know (1)
58. Criticisms/suggested improvements initial briefing on the evening before the navigation exercise did give enough detail - some confusion
Whilst in action on the hill some students (especially those with less experience) found the observation / assessment system was time-consuming and difficult to apply, using up valuable navigation time.
59. Some students did not like the A to D marking system as they felt it was too inflexible and had formal assessment implications.
Several experienced students (some of whom hold the ML Award and higher awards) felt that this exercise was far better suited to participants who already have some mountain navigation skills, and that beginners would benefit from a more structured and supervised teaching approach.
60. Conclusions More experienced students found the Four Ds model gave them a useful framework into which to fit their navigational skills
novice mountaineers found the Four Ds model a useful framework but one which was limited by their background. Some felt inhibited by a lack of confidence.
value of analytical observation being a two way process, in which observer and observed were learning, had not been anticipated beforehand
61. Less experienced mountaineers amongst the students tended to find the peer reviewing exercise time consuming and frustrating and felt that they needed more guidance from staff
62. Further Research Feb-Mar 2004: Leadership Styles
2005 ? Use of pocket GPS (Global Positioning Systems) in Mountain Leader Training
64. Thank you for your attention
