Controversies in Cognition

1. Controversies in Cognition Evaluation of technology use ‘in the wild’ Dr. Danaë Stanton Fraser

2. Controversies 2006 Lectures • Space. Can we simulate it effectively? • Spatial Cognition: learning what has been 'learnt' from cognitive maps. • Laboratory versus Field: the Evaluation Debate • Estimating time-to-collision: can we accurately estimate when we will reach an object or when a moving object will reach us? • Evaluating ‘in the wild’ • Aiding Mobility: are electronic and environmental devices sufficient for aiding mobility in the mobility impaired?

3. Controversies 2006 Introduction • Traditional technologies in subservient role to Psychology • Methods of recording and storing data • e.g. desktop computers used to time reactions • Domains of data gathering • e.g. digital video recording enables repeatable analysis of behaviour • Methods of conducting experiments • software provides reproducible experiences during experiments • Access to people • Larger numbers of participants possible in on-line surveys or questionnaires

4. Controversies 2006 Evaluating in the Wild • New technologies also provide an interesting domain for study • New possibilities for collaboration and changing methods of communication (e.g. telephone, email, SMS, videoconferencing, …) • The Shared Desktop • Sensors and Context • Mobile Systems

5. Controversies 2006 Working with schools (1) • Education • Working with schools versus testing in schools • The practicalities: consent, ethics, relationship with user group, OFSTED, SATS • Designing within the physical, social and organisational constraints of a real classroom • Children as partners versus informants • Design methods

6. Controversies 2006 Working with schools (2) • How to gain access to schools • Ethics approval – school policies • Consent forms to parents • Through school • Include use of video recordings (research papers now often go on the web) • Teacher’s approval

7. Controversies 2006 Designing with or for? • Different ways of designing and working with user groups (see Druin, 2002) • Design partners • Informants • “Subjects”/Participants

8. Controversies 2006 Participatory design • Involve users as members of the design team from the start • Methods of communication • brain (and body) storming • pen and paper interface walkthroughs • paper/cardboard mockups • early prototypes • emerged from Scandinavia

9. Controversies 2006 Designing with Children • Children and teachers involved in the design process. • Work intensively with teachers and children in school to design and develop technologies to be integrated into the classroom? • Interdisciplinary team? Those developing the technology should also go to the school!

10. Controversies 2006 Designing with Children • Integrating with the National Curriculum? • The physical nature of the classroom means that children are continually divided into small groups • Iterative design sometimes slower development of technology but a more ‘integrated’ and ‘usable’ product.

11. Controversies 2006 Traditionally… • Computer in the corner of the classroom • Computer lessons not linked to domain • Individual use of machine, or • Possibility for input < number of children per device • Children and teachers not actively involved in the design of the technology being used

12. Controversies 2006 New Technologies: The Shared Desktop • Working in pairs and groups can have advantageous effects on learning and development (Rogoff, 1990; Wood and O’Malley, 1996) • Role of the computer unique in the way that it can structure collaborative activity (Littleton, 1999) • Traditionally computer hardware and software designed with one user in mind

13. Controversies 2006 • KidStory project - Developing collaborative storytelling technologies for children aged 5-7 • Learning in class is a social activity - focus on group work • Designing within the physical, social and organisational constraints of a real classroom

14. Controversies 2006 An evaluation study: Collaborative behaviour around the computer • An exploratory study to examine the effect of multiple mice on children’s dialogue and interaction

15. Controversies 2006 • Single Display Groupware - allows to co-located users to interact with a system simultaneously • Inkpen et al (1995, 1997, 1999) found significant learning improvements - higher levels of activity and less off task behaviour • Abnett et al (2001) gender effects.

16. Controversies 2006 Method • Participants - 24 children from an infant school in Nottingham. Aged 6-7 years. • Apparatus - KidPad a shared 2D drawing tool with a zooming interface (Druin et al, 1997). • Task - Creative task carried out by pairs of children using one or two mice.

17. Controversies 2006 • Procedure Recreate a poem in KidPad Children encouraged to work together 20 minutes Video capture of the computer screen and the children were mixed An analysis of the process of collaboration

18. Controversies 2006 Examination of behaviour in depth • Coding scheme developed to capture types of talk, physical interaction and their relationship with the on screen product • Qualitative analysis • The development of the car in the poem analysed for each pair (5minutes)

19. Controversies 2006 Characteristics and Behavioural styles observed Interaction with 2 mice Common themes Verbalisation of action Little reciprocity or elaboration of ideas Active division of tasks - working in parallel Still cases of dominant behaviour by one partner

20. Controversies 2006 Results Use of two mice: Greater degree of engagement in task more total time spent on creation symmetry of mouse use

21. Controversies 2006 Asymmetry with one mouse

22. Controversies 2006 Symmetry with 2 mice

23. Controversies 2006 Video

24. Controversies 2006 Interaction with one mouse More of a mix of behaviours Good collaboration - long discussion of ideas, reciprocity followed by input of joint ideas, conflict followed by compromise Conflict not resolved, high degree of negativity about others work Domination by one partner

25. Controversies 2006 New Technologies: Mobile and Wireless Outside the school: the fieldtrip • field trip ‘with a difference’ • Taking technology outdoors • Making the invisible visible • Carry out collaborative discovery and reflection • To stimulate scientific enquiry

26. Controversies 2006 The Ambient Wood Project • small groups of children using mobile technologies outdoors to support scientific enquiry about the biological processes taking place in a wood. • One of the devices used, a probe tool, contained sensors enabling measurement of the light and moisture levels within the wood. A small screen was also provided which displayed the readings using appropriate visualisations.

27. Controversies 2006 Handhelds to make the invisible visible • Measurements of light and moisture at different locations were displayed on a PDA in pictorial format. • Mobile devices used to receive location-specific information.

28. Controversies 2006 Video

29. Controversies 2006 Key findings Analysis of the patterns of interaction revealed: • The probe engendered exploration, the generation of ideas (about where to probe in order to get different readings, or to see readings around particular plants). • Children made links between their readings, for example, comparing readings taken by the same species of plant, but in different locations. • Children made predictions about readings they might expect in particular locations, for example, one pair predicted a moist reading because there was lots of moss. • Many also drew conclusions about the general physical state of the woodland, and how this related to the environment and the organisms found on the basis of their probe readings.

30. Controversies 2006 New Technologies: Mobile and Sensors • Integrating the outdoors with back in the classroom • To explore how emerging networking technologies can enhance science education • Hands-on approach to learning science in schools • Children learn about presence and impact of pollution • Encourage an understanding of the scientific process • Support collaborative activity between different schools and with scientists

31. Controversies 2006 Activities Children as active scientists • 2 schools involved • Primary in Nottingham • Secondary in Brighton • Sessions to familiarise children with pollution • Children use mobile carbon monoxide sensors to measure pollution levels in their local environment • Use software tools to analyse their data in the classroom and to share with others

32. Controversies 2006 Mobile Sensors • Children can collect pollution data • Mobile sensor records and gives instant feedback

33. Controversies 2006

34. Controversies 2006 Analysing data Video replay Textual annotation Interest point Time series

35. Controversies 2006 Connectivity, schools and scientists • Children in same school • Scientific Teams • Children in different schools • Connectivity, Community • Children and scientists • Learning from the Experts

36. Controversies 2006 Summary of results Results of video analysis and teacher interviews suggest that this context-inclusive approach is significant for three reasons: • Firstly, it allows individuals to reflect on method as part of data collection. • Secondly it provides an aide-memoir to groups who have collected data together in interpreting results. • Thirdly, it allows new participants who have engaged in similar processes to understand new perspectives on their own and others’ data.

37. Controversies 2006 Controversies • Should we be as controlled/experimental as possible/as the pre-determined factors allow? • Is the only benefit of exploration and engagement in the field to provide ideas for experiments? • Is lab-based work appropriate in studying field-based use?

38. Controversies 2006 References (1) Abnett, C., Stanton, D., Neale, H and O’Malley (2001) The effect of multiple input devices on collaboration and gender issues. In the Proceedings of European Perspectives on Computer-Supported Collaborative Learning (EuroCSCL) 2001, March 22-24, Maastricht, the Netherlands. P.29-36. Druin, A., Stewart, J., Proft, D., Bederson, B., Hollan, J. (1997). KidPad: A design collaboration between children, technologists, and educators. Proceedings of  CHI’97, Atlanta, GA. Druin, A. (2002). The Role of Children in the Design of New Technology. Behaviour and Information Technology, 21(1) 1-25. Inkpen, K., Booth, K.S., Klawe, M., and Upitis, R. (1995). Playing Together Beats Playing Apart, Especially for Girls. Proceedings of Computer Supported Collaborative Learning (CSCL) '95. Lawrence Erlbaum Associates, 177-181. Inkpen, K. M., Booth, K. S., Klawe, M., & McGrenere, J. (1997). The Effect of Turn-Taking Protocols on Children's Learning in Mouse-Driven Collaborative Environments. In Proceedings of Graphics Interface (GI 97) Canadian Information Processing Society, pp. 138-145. Inkpen, K.M., Ho-Ching, W., Kuederle, O., Scott, S.D. & Shoemaker, G.B.D. (1999) ‘This is fun! We’re all best friends and we’re all playing’: Supporting children’s synchronous collaboration. In Proceedings of Computer Supported Collaborative Learning (CSCL99) (eds. C.M. Hoadley & J. Roschelle) pp. 252–259. Lawrence Erlbaum, Hillsdale, NJ.

39. Controversies 2006 References (2) Littleton, K. (1999). Productivity through interaction: An overview. In K. Littleton and P. Light (Eds.) Learning with Computers: Analysing productive interaction. Routledge. London p.179-194. Price, S., Rogers, Y., Stanton, D. and Smith, H. (2003). A new conceptual framework for CSCL: Supporting diverse forms of reflection through multiple interactions. In Proceedings of Computer Support for Collaborative Learning. (CSCL) 2003, Kluwer, pp. 513-523. Rogers, Y, Price, S., Randell, C, Stanton Fraser, D., Weal M. and Fitzpatrick, G. (2005). Ubi-learning: Integrating Indoor and Outdoor Learning Experiences. Communications of the ACM. January 2005/Vol. 48, No. 1 Rogoff, B., Apprenticeship in Thinking: Cognitive Development in Social Context. New York: Oxford University Press, 1990. Stanton, D., Neale, H. and Bayon, V. (2002) Interfaces to support children's co-present collaboration: multiple mice and tangible technologies. Computer Support for Collaborative Learning. (CSCL) 2002. ACM Press. Boulder, Colorado, USA. January 7th-11th.p.342-352 Stanton, D. and Neale, H. (2003). Collaborative Behaviour around a computer: the effect of multiple mice on children’s talk and interaction. Journal of Computer Assisted Learning (JCAL), Blackwell, Vol. 19, no. 2, pp. 229-239. Stanton Fraser, D., Smith, H., Tallyn, E., Kirk, D., Benford, S., Rowland, D., Paxton, M., Price S and Fitzpatrick G. (2005). The SENSE project: a context-inclusive approach to studying environmental science within and across schools. Computer Supported Collaborative Learning (CSCL 2005). Taiwan. May. Tallyn, E., Stanton, D., Benford, S., Rowland, D., Kirk, D., Paxton, M., et al. (2004). Introducing eScience to the classroom.Proceedings of the UK e-Science All Hands Meeting, EPSRC, pp. 1027-1029. Wood, D., & O'Malley, C., Collaborative learning between peers: An overview. Educational Psychology in Practice, 11(4), 4-9, 1996