Learning Technology Research

1. Learning Technology Research Peter Goodyear Quality Research: fitness for purpose (Re)framing relations between technology and learning Capacity-building, collaboration, etc http://www.slideshare.net/petergoodyear90/health-elearning-rg-may-2014

2. Educational research qualitydoes not have a strong reputation “While there was world leading research in most subfields, at the lower end there was insufficient building upon other work, including testing hypotheses/research questions in new contexts. The contrast between the innovative and the pedestrian was especially acute in research in HE, assessment, early years, adult education, and in subject-discipline focused work, including ICT. Small scale work sometimes reflected inward referencing sub-communities with insufficient awareness of the generic mainstream or of relevant international developments.” UK RAE 2008 Subject Overview Report of the Education Panel, p3 http://www.rae.ac.uk/pubs/2009/ov/

3. Educational research quality application-focussed R&D has extra problems 1) Institutions can be risk averse when selecting research outputs to be assessed for quality (eg RAE, peer-review in ERA); user-focused outputs under-represented in UK 2008 RAE 2) Best user-oriented outputs: “sophisticated and complex ideas and data were vividly and eloquently communicated by their authors” Weaker outputs: ”valuable material was expressed in an unnecessarily arcane and convoluted way” 3) “user audiences tend to be impressed by work that consciously builds on what has been done before – the systematic review and research synthesis being prime examples of cumulative work” – much of the weaker work made insufficient reference to recent, relevant research; missed opportunities to integrate knowledge 4) “given that many researchers in education come to research from a teaching career or an academic career outside the discipline of education, we think that it might be advisable in any future exercise to make an even clearer distinction between pedagogical research in higher education and descriptive or anecdotal accounts of teaching developments and evaluations

4. Resolving the tension between applied and pure researchResearch in ‘Pasteur’s Quadrant’(Stokes, 1997) Alan Schoenfeld UC Berkeley (1999) Considerations of use (practical application etc)? Quest for fundamental understanding (theoretical/ scientific advances, etc)?

5. Improving the quality of educational research(in Pasteur’s Quadrant) Considerations of use (practical application etc)? Quest for fundamental understanding (theoretical/ scientific advances, etc)? Quality (fitness for purpose)

6. Unpacking ‘quality’ Lack of attention to these issues has undermined general perceptions of the usefulness of educational research • Considerations of use (practical application etc)? • Use by whom? • For what purposes? • How do they work? • What kinds of knowledge can make a difference to their work? • Resolvable. • Quest for fundamental understanding (theoretical/ scientific advances, etc)? • What needs explaining? • What does an adequate explanation look like? • What kinds of facts, principles, models, theories, etc are suitable for our domain(s) of study? • Widely (and wildly) contested – make an explicit, considered choice & justify it.

7. Unpacking ‘quality’ Lack of attention to these issues has undermined general perceptions of the usefulness of educational research • Considerations of use (practical application etc)? • Use by whom? • For what purposes? • How do they work? • What kinds of knowledge can make a difference to their work? • Empirically resolvable. Research users are local: key actors in an evolving ecosystem – research informs iterative improvements to course, curriculum, pedagogy, environment, tools & resources etc; actionable knowledge, contextually sensitive; cf formative evaluation, participatory design Research users are distant/global: the research & (re)design loop is mediated by publication in journal articles etc; generalisation; principled/declarative knowledge;

8. Types of Educational Research (IES/NSF) Foundational (Early Stage / Exploratory) Foundational Early-stage/exploratory Design & development research Impact research (What works) Efficacy research Effectiveness research Scale-up research ‘Gold Standard’ Methods for #5 & #6 (such as RCTs) are inappropriate for foundational/early stage research. Bench -> Bedside Lab -> Learnplace Translation processes are researchable too Source: this and next two slides adapted from National Academy of Engineering and National Research Council (2014) STEM integration in K-12 education: Status, prospects, and an agenda for research, Washington DC, The National Academies Press. pp138-40. Much more extensive version in IES/NSF (2013) Common guidelines for education research and development. Washington DC. http://ies.ed.gov/pdf/CommonGuidelines.pdf

9. Foundational/Early-Stage Contributes to core knowledge in education (eg basic understandings of teaching and learning, cognition, competence, functioning of educational systems and learning environments, etc) Foundational: provides fundamental knowledge that may be relevant to improving learning outcomes; theories of learning, cognitive processes etc Early-stage/exploratory: Finding relationships/patterns in educational variables; exploring logic of relationships in ways that may inform subsequent interventions Design & development research: Develops and tests candidate solutions to educational problems; may be as much for refinement of theory or improved understanding the nature of the problem as for demonstrating the value of the solution

10. Impact research Contributes to evidence of impact; reliable estimates of the ability of a fully-developed intervention (etc) to achieve intended outcomes; methodological imperatives to precisely specify the intervention, the intended outcome measure, reduce bias) Efficacy: testing under ideal/enhanced conditions (eg with high level of support from researchers etc who developed the intervention) Effectiveness: examines effectiveness in ‘normal’ circumstances (minimal developer involvement) Scale-up: examines effectiveness across a wide range of populations, contexts, cicumstances etc

11. (Re)framing relations between technology and learning From gadget-based acceptance testing to the orderly evolution of learning systems

12. Activity-centered analytical framework Physically Situated Artifacts, tools, texts, etc (Emergent) Activity Outcomes Tasks Dyads, groups, teams; roles; divisions of labor Socially Situated (Goodyear & Carvalho, 2014,  p. 59)

13. Activity-centered analytical framework Goal-directed action (intentions formed in the mind precede and direct selection of tools, actions etc) Embodied cognition, extended mind, intermingling of mind-body-world (tool-using action brings forth intention) Physically Situated Artifacts, tools, texts, etc (Emergent) Activity Outcomes Tasks Dyads, groups, teams; roles; divisions of labor Socially Situated (Goodyear & Carvalho, 2014,  p. 59)

14. Technology-aided learning comes naturally “Human-machine symbiosis, I believe, is simply what comes naturally. It lies on a direct continuum with clothes, cooking (‘external, artificial digestion’), bricklaying and writing. The capacity to creatively distribute labour across biology and the designed environment is the very signature of our species,and it implies no real loss of control on our part. For who we are is in large part a function of the webs of surrounding structure in which the conscious mind exercises at best a kind of gentle, indirect control. ” (Andy Clark, 2003, p174, emphasis added)

15. Metaphors for learning • Learning as acquisition • Learning as participation • Learning through participation (e.g. to acquire ...) • Learning to participate (e.g. in a valued, ongoing practice) • Learning as knowledge creation • Learning as creating epistemic tools/environments

16. Activity-centered analytical framework Metaphors for learning Learning to participate involves understanding the properties of tools (etc); incorporating them into your activities (instrumental genesis); using them to create new tools

17. Outcomes e.g. cognitive; psychomotor; affective identity, enculturation, engagement in valued practices, etc Study situation Intrinsic feedback from self-monitoring; over various timescales Extrinsic feedback via others; over various timescales Activity e.g. mental; physical problem-solving; reflective with/without others; with/without tools/artefacts heavily situated or abstracted, etc People e.g. friends, workmates individually; collectively dyads, groups, teams, communities Tools, artefacts and other resources in the material-digital world e.g. books; wikis, email; LMS/VLEs; seminar rooms, etc Maintenance Products Tasks Macro-meso-micro in scale e.g. dissertation project; essay; lab report; answering a question embedded meso-scale assessed tasks as key devices Activity-centered analysis

18. Outcomes Activity People Tools, artefacts Tasks Activity-centered analysis The nature of valued learning outcomes Reliable links between activities and outcomes The ergonomics of learning environments Effects of grouping strategies Factors affecting the translation of tasks into activities (deep, surface & strategic learning…)

19. Shifting focus from the affordances of single tools/artefacts to analysing and influencing (? as a participant) complex evolving meshworks of people and things

20. Support & Collaboration IISME: Innovations in Science & Maths Education CoCo: Researching combinations of new pedagogy & new technology CHAI: Human-Computer Interaction Research Latte: Learning and Affective Technologies ?CPC Science of Learning Science node: methods for researching learning in biomedical sci STL: Sciences & Technologies of Learning SyReNs network focus of researching in STEM education & professional learning http://sydney.edu.au/research/stl/http://blogs.usyd.edu.au/stl/ “CRISTL”: Research Capacity Building & Research-Informed Educational Innovation not an overarching structure; facilitating networking peter.goodyear@sydney.edu.au

21. Recommended reading • Sfard, A. (1998) On two metaphors for learning and the dangers of just choosing one. Educational Researcher, 27, 4-12. • Paavola, S., Lipponen, L. & Hakkarainen, K. (2004) Models of innovative knowledge communities and three metaphors of learning. Review of Educational Research, 74, 557-576. • Overdijk, M., Diggelen, W., Kirschner, P. & Baker, M. (2012) Connecting agents and artifacts in CSCL: Towards a rationale of mutual shaping. International Journal of Computer-Supported Collaborative Learning, 7, 193-210. • Affordances; Structuration; Instrumental genesis • Lonchamp, J. (2012) An instrumental perspective on CSCL systems. International Journal of Computer-Supported Collaborative Learning, 7, 211-237. • Malafouris, L. & Renfrew, C. (Eds.) (2010) The cognitive life of things: Recasting the boundaries of the mind, Cambridge, McDonald Institute for Archaeological Research, University of Cambridge.

22. Further reading: connecting physical setting to human activity Malafouris, L. & Renfrew, C. (Eds.) (2010) The cognitive life of things: Recasting the boundaries of the mind, Cambridge, McDonald Institute for Archaeological Research, University of Cambridge. Malafouris, L. (2013) How things shape the mind: A theory of material engagement, Cambridge, MA, MIT Press. Ingold, T. (2011) Being alive: Essays on movement, knowledge and description, Abingdon, Routledge. Clark, A. (2008) Supersizing the mind: Embodiment, action, and cognitive extension, Oxford, Oxford University Press. Clark, A. (2003) Natural-born cyborgs: Minds, technologies, and the future of human intelligence, Oxford, Oxford University Press. Sterelny, K. (2012) The evolved apprentice: How evolution made humans unique, Cambridge MA, MIT Press. Sterelny, K. (2003) Thought in a hostile world: The evolution of human cognition, Oxford, Blackwell. Kirsh, D. (2013) Embodied cognition and the magical future of interaction design. ACM Trans. Comput.-Hum. Interact. 20, 1-30.

23. Probably won’t use slides after this point except if needed for discussion Follow up: Goodyear, P. & Carvalho, L. (2013) The analysis of complex learning environments. In Beetham, H. & Sharpe, R. (Eds.) Rethinking pedagogy for a digital age: Designing and delivering e-learning. RoutledgeFalmer.

24. Clark: self-engineering “We do not just self-engineer better worlds to think in. We self-engineer ourselves to think and perform better in the worlds we find ourselves in. We self-engineer worlds in which to build better worlds to think in. We build better tools to think with and use these very tools to discover still better tools to think with. We tune the way we use these tools by building educational practices to train ourselves to use our best cognitive tools better. We even tune the way we tune the way we use our best cognitive tools by devising environments that help build better environments for educating ourselves in the use of our own cognitive tools (e.g. environments geared toward teacher education and training). Our mature mental routines are not merely self-engineered: they are massively, overwhelming, almost unimaginably self-engineered.” Clark, A. (2008) Supersizing the mind: Embodiment, action, and cognitive extension, Oxford, Oxford University Press, pp59-60.

25. 1 way communication checking pulse visual search 2 way communication administering painkillers bandaging position and route finding traversing steep ground evacuating casualty worrying about damaging iPad

26. Analysis and design of complex learning environments: sociomaterial perspectives “…knowledge generation … [is] … a joint exercise of relational strategies within networks that are spread across space and time, and performed through inanimate (e.g. books, mobile phones, measuring instruments, projection screens, boxes, locks) as well as animate beings in precarious arrangements. Learning and knowing are performed in the processes of assembling and maintaining these networks, as well as in the negotiations that occur at various nodes comprising a network… Things – not just humans, but the parts that make up humans and non-humans – persuade, coerce, seduce, resist and compromise each other as they come together.” Tara Fenwick et al., (2011, 10) Emerging approaches to educational research: tracing the sociomaterial

27. Successful participation in the exercise involves: learning to use each tool, at least with sufficient fluency to be able to act according to the established protocols, but ideally with a level of automaticity that binds tool and action in a smooth flow integrating the use of the tools into a web of activity, involving smooth effective action, co‐ordination with others, focus on the priority goals, etc. turning the individual and aggregate experiences of the exercise into learning that lasts The point of the exercise is not to master the individual tools but to participate in the construction of a co‐ordinated web of activity that can result in a successful rescue, minimising danger to participants, and leaving traces (in some kinds of memory) that mean doing something like this again will not feel entirely new….