Integrated assessment of disciplinary, personal and interpersonal skills in a design-build course

1. Integrated assessment of disciplinary, personal and interpersonal skillsin a design-build course Kristina Edström, Khalid El Gaidi, Stefan Hallströmand Jakob Kuttenkeuler June 8, 2005

2. Integrated assessment of disciplinary, personal and interpersonal skillsin adesign-build course Kristina Edström, Khalid El Gaidi, Stefan Hallströmand Jakob Kuttenkeuler June 8, 2005

3. Integrated assessment of disciplinary, personal and interpersonal skillsin a design-build course Kristina Edström, Khalid El Gaidi, Stefan Hallströmand Jakob Kuttenkeuler June 8, 2005

4. The design-build course • Idea to realisation (verifiable!) • Large project groups (8-15 students) • Final year • Full year - 1/4 time • Open-ended project specifications

5. Project task example Bring 2 kg of payload up in the air and keep it there using the sun as the only source of power + a few constraints

6. Project examples 2001-2002 Solar powered aircraft 2002-2003 Waterbike 2003-2004 Subskimmer

7. Solar powered aircraft

8. The course format • Project based • Some activities predefined and fixed – others to be defined • "A smörgåsbord syllabus" – different students do different things: • Conceptual analysis • Deep analysis • Project management • Manufacturing • Experiments • PR & sponsoring • Planning and follow-up • … Typically all students do most of the things but no student does everything

9. Course goals After the course the participant is expected to be able to work efficiently as an engineer in a project environment, i.e.: • take personal responsibility for small tasks and the project as a whole • analyse technical problems from a holistic point of view • handle technical problems which are incompletely stated and subjects to multiple constraints • develop strategies for systematic choice and use of availableengineering methods and tools • make estimations and appreciate their value and limitations • make decisions based on acquired knowledge • communicate engineering – orally, in writing and graphically

10. Integrated learning What does it mean to • communicate engineering – orally, in writing and graphically ? • to describe and present ideas • to develop ideas through collaborative sketching and engineering reasoning • if necessary, to argue for or against conceptual ideas and solutions • to communicate technically, both with experts and laypersons • to nurture confidence in expressing oneself within a certain field of work Communicating engineering is contextual

11. Assessment challenges and approach We want to assess individual achievements in a group setting How?! As personal and interpersonal skills are explicitly addressed personal and interpersonal skills should be assessed! We could only assess what we could observe or measure but here the main part of the work is carried out behind the scenes! In order to assess the work behind the scenes one has to either be there or ask those who are! The students’ trust in us is sometimes worrying…

12. Assessment strategy Introduction Faculty • communicate course goals • instruct students to collect evidence (portfolios) • instruct students to log own and others’ activities Students • express personal goals • plan own time • declare anticipated working experiences START END

13. Assessment strategy Mid-course poll Faculty • recap course goals • teach giving/receiving feedback Students • write narratives • read narratives, write feedback, suggest peer grades • read feedback, write reflective documents • revisit/revise personal goals • follow up with reflective exercises START END

14. Assessment strategy Final assessment Faculty • recap course goals • recap reflections from poll Students • write narratives • write feedback, suggest grades • follow up with discussions on the contribution from each student • follow up with reflective exercises • write reflective documents START END

15. Grading The grades are finally set based on a holistic assessment of • portfolios (reports, protocols, presentations, sketches, hardware, …) • with respect to course goals • given feedback • received feedback + recommended grades • reflective documents • participation and guided by • logged time • continuous observations

16. Some general observations • Students’ attitudes to knowledge are challenged • Applied knowledge differs from theoretical knowledge • Decision making is new to students • Distinction between project goals and course goals is essential • Peer assessment is delicate to compose – not enough to use the right instruments, one has to play them careful too

17. Conclusions and final remarks • Open-ended project implies limited number of predetermined activities • assessment scheme should be generic • In order for peer assessment to be useful and constructive • students must be comfortable in the process • students must rely on the process • The experience from developing the peer assessment scheme is that • it is not only what you do that matters but also • how you do it and • how you present it to the students