Teaching as a design science: Designing and assessing the effectiveness of the pedagogy in learning technologies

1. Teaching as a design science: Designing and assessing the effectiveness of the pedagogy in learning technologies Diana Laurillard London Knowledge Lab Institute of Education eLearning Forum Asia 2013 Hong Kong Baptist University 29-31 May 2013

2. The Challenges to Higher Education The issues • Global demand for HE • The aims of HE • The roles of TEL • Modelling learning technologies • Teachers as innovators

3. The policy context • Teachers to have the capacity to plan and deliver ICT-enriched learning experiences for students to become self-directed and collaborative learners... (Singapore ICT Masterplan, 2008) • Teacher capacities will be developed in instructional design, selection and critical evaluation of digital content, and strategies for effective use of digital content to enhance student learning. (India Government, 2009) • Professional educators connected by technology to empower, and inspire effective teaching (US Plan 2010) • … greater prioritisation of teaching partnerships between technologists, learning support specialists and academics, and an end to the ‘not invented here’ syndrome… Good practice must also be shared. (HEFCE OLTF, 2011) • Professional development is needed for all school staff to support the process of ICT integration in schools(UNESCO, ICT in Primary, 2012)

4. The global demand for HE The new UNESCO goals for education: • Every child completes a full 9 years of free basic education … • Post-basic education expanded to meet needs for knowledge and skills … (Draft for UNESCO post 2015 goals) By 2025, the global demand for higher education will double from 100m to ~200m per year, mostly from emerging economies(NAFSA 2010) - Implying significant teacher training needs for HE 25:1 student:staff ratio = 4m new teachers??

5. The aims and purpose of HE The purposes of higher education: • To inspire and enable individuals to develop their capabilities to the highest… • To increase knowledge and understanding for their own sake… • To serve the needs of an adaptable, sustainable knowledge-based economy… • To play a major role in shaping a democratic, civilised, inclusive society (NCIHE, 1997) How could mass HE achieve that nurturing of the individual, while reducing the current 25:1 student:staff ratio for student support?

6. An example: The Duke MOOC Bioelectricity: A Quantitative Approach Taught in class for over 20 years Experimental move to a free and open MOOC 12,000 students enrolled from >100 countries • 8 weeks long • 97 ~6 min videos • 22 GB of data • 1052 files • 18 gradedexercises, including a peer-gradedwritingassignment and final exam (DukeUniversity 2013)

7. The Duke MOOC Not for undergraduates Enrolled students Potential undergraduates

8. The Duke MOOC Not for the faint-hearted Comparable with normal online u/g courses

9. Basic MOOCs vs the Duke MOOC: Comparing the learning experience Basic: 8 weeks, providing 50 hours learning time, no teacher support • Duke: 8 weeks, providing 50 hours learning time including support: • Videos and pdfs • Quizzes • Wiki • Peer discussions • Peer grading • Tutored discussions • Summative assessment High on prep time Zero contact for 42 hours Low on prep time High contact for 8 hours 420 hours to develop materials and course design 200 hours to support ~500 students for 8 hours = 20:1 student:staff ratio Report at http://bit.ly/ZRMbjp

10. Comparing teacher hours for a basic MOOC and the Duke MOOC (48 hour course) Total teacher time Teacher support time rises to 2000 hours for 5000 students. 2000 hours = 1 year of a tutor for a 5 credit course. = 24 FT tutors for 120 credit course. 200 Prep time Prep time = 420 hrs

11. Modelling the benefits and costs • We need to understand the pedagogical benefits and teacher time costs of online HE • What are the new digital pedagogies that will address the 25:1 student support conundrum? • Who will innovate, test, and build the evidence for what works at scale online? TEACHERS!

12. Pedagogies for supporting large classes Concealed MCQs The virtual Keller Plan The vicarious master class Pyramid discussion groups Conceal answers to question Ask for user-constructed input Reveal multiple answers Ask user to select nearest fit Introduce content Self-paced practice Tutor-marked test Student becomes tutor for credit Until half class is tutoring the rest 240 individual students produce response to open question Pairs compare and produce joint response Groups of 4 compare and produce joint response and post as one of 10 responses... 6 groups of 40 students vote on best response Teacher receives 6 responses to comment on Tutorial for 5 representative students Questions and guidance represent all students’ needs

13. Digital pedagogies for scaling up to higher student:staff ratios without losing quality Concealed MCQs The virtual Keller Plan The vicarious master class Pyramid discussion groups Like MCQs, no teacher support More challenging learning experience Little teacher support Every learner receives 1-1 support One teacher guides 5 students They represent all students’ needs Teacher reacts to 6 student outputs All students contribute, discuss, vote

14. How might technology help teachers? First things first – how do students learn?How does technology help?Tools for teachers to build pedagogic knowledgeTeachers as design scientists…

15. The learner learning Acquiring Teacher concepts L C L C Learner concepts Inquiring Modulate Generate L P L P Learner practice Learning through acquisition, instruction Learning through inquiry

16. The learner learning Teacher concepts Learner concepts L C L C Modulate Modulate Generate Generate Task Feedback L P L P Learner practice Learning environment Actions Learning through practice with meaningful intrinsic feedback

17. The learner learning Acquiring Ideas, questions Teacher concepts Learner concepts Peer concepts L C L C Ideas, questions Inquiring Modulate Modulate Modulate Generate Generate Generate Task/Feedback Outputs L P L P Learning environment Learner practice Peer practice Outputs Actions Learning through discussion from peers’ ideas, questions Learning through sharing from peers’ practice

18. Representing types of learning Acquiring Teacher communication cycle Peer communication cycle Teacher concepts Learner concepts Peer concepts L C L C Inquiring Discussing Producing Modulate Modulate Generate Generate Peer modelling cycle Teacher modelling cycle Sharing L P L P Learning environment Learner practice Peer practice Practising The teacher needs to design forall these types of learning

19. Books, lectures, papers, websites, videos, AR, podcasts… Deploying learning technologies Tutorials, seminars, small groups, online forums, social media… Libraries, journals, repositories, databases, Web… Book… Podcast Teacher communication cycle Peer communication cycle Teacher concepts Learner concepts Peer concepts L C L C Library… Web Tutorial… Facebook Essay… Model Modulate Modulate Generate Generate Peer modelling cycle Teacher modelling cycle Project… VLE L P L P Learning environment Learner practice Peer practice Lab… Simulation Labs, exercises, problems, projects, serious games, models, simulations… Essays, designs, performance, programs, videos, ppts, digital assets, models, e-portfolios… Project groups, teamwork, Google groups, wikis, VLEs… The teacher needs to use all types of learning technology within the Conversational Framework

20. Collecting learning analytics What accessed in what sequence Social interaction patterns Questions asked Book… Podcast Teacher communication cycle Peer communication cycle Teacher concepts Learner concepts Peer concepts L C L C Library… Web Tutorial… Facebook Essay… Model Modulate Modulate Generate Generate Peer modelling cycle Teacher modelling cycle Project… VLE L P L P Learning environment Learner practice Peer practice Lab… Simulation Tracked inputs, reaction times, sequence Analysis of essay content, quiz scores, game scores, accuracy of model Tracked group outputs, peer assessment

21. Forms of TEL/online learning activities Learning activities for online courses • Guided TEL resources (model, AR) • Access to expositions (lecture videos) • Automated grading (MCQs, models) • Readings (pdfs) • Guided collaboration activities (wiki, AR) • Peer group discussion (forum) • Peer grading against criteria (share) • Tutored discussion (forum) • Tutor feedback (assignments) • Tutor-based assessment (exams) What are effective ways of combining and sequencing these activities for learning and formative assessment, that will help students achieve the intended outcomes?

22. Tools for teachers as learning designers Teachers as designers need the tools for innovation To find or create new ideas Adopt Adapt Test To collect learning analytics Redesign Analyse Publish Creating knowledge about effective online pedagogies for specific learning outcomes Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York and London: Routledge.

23. Tools for teachers as learning designers

24. Tools for teachers as designers - PPC A library of patterns to inspect

25. Tools for teachers as designers - PPC Each pattern is designed to deliver a specific type of learning outcome Colour-coded text identifies content parameters Black text expresses pedagogy design

26. Transferring the pedagogy across topics - PPC The Pedagogical Patterns Collector

27. Transferring the pedagogy across topics - PPC The Pedagogical Patterns Collector

28. Transferring the pedagogy across topics - PPC

29. Teacher adopts and adapts a design Export to Word [Moodle] Check the feedback on the overall distribution of learning activity Add link to an OER, e.g. a digital tool for practice Read, Watch, Listen Inquire Discuss Practice Share Produce Represent the teacher as present or not Adjust the type of learning activity. Edit the instructions. Specify duration of the activity in mins Fully specifies the pedagogy for others to adopt, adapt, test, and share

30. Design for Med students in PPC Explain how to optimise the inputs to a patient simulator to achieve the ideal blood pressure With your partner select different inputs to the patient simulator – can you improve on your previous results?

31. Export to Moodle for Med students

32. Re-design for Business students in PPC Explain how to optimise the inputs to a business model to achieve the optimal cash flow Explain how to optimise the inputs to a patient simulator to achieve the ideal blood pressure With your partner select different inputs to the business model – can you improve on your previous results? With your partner select different inputs to the patient simulator – can you improve on your previous results?

33. Export to Moodle for Business students Export to Moodle for Med students The process of professional collaboration: Find effective patterns for a learning objective – adopt, adapt, improve – export to Moodle – test with students – improve – test - publish

34. Comments on the PPC • [The pie-chart] is one of the most useful features … it gives a good overview of the balance between different learning experiences • I rarely consider how the students' time is apportioned … it's good to be made to think about this. • Seeing how the sessions are shaping up in such a visual medium …. would probably make me think more carefully about providing a mix of activities

35. Tools for teachers as learning designers 教学设计库 PPC 浏覽器 欢迎 浏覽器 提供了一些通用的教学设计及其相关的实例，你可以参考或改编这些设计用于自己的教学设计之上 这套教学设计库工具包使教师能够分享他们好的教学理念。其目的是帮助学科教师了解某个特定教学设计可以如何成功地应用于不同的教学主题 。系統中有教学设计样本供浏览和编辑，你也可以从头开始自行设计。这是由ESRC- EPSRC资助的TLRP-TEL研究项目之‘教学设计支持环境’的一项成果。 设计师 PPC设计师 展示教学设计模板来帮助你描述自己某节课的教学理念（如学生准备、课堂活动和家庭作业） [Chinese version coming soon…]

36. Modellinglearning experience and teacher workload • With a computational model, we can estimate the effects of the design decisions on quality of learning experience AND teaching cost: • consequences for the pedagogical benefits • comparative costs of teachers’ workload • Then check actual outcomes using targeted learning analytics

37. Enhancement: Modelling the pedagogic benefits A computational representation can analyse how much of each activity has been designed in Categorised learning activities Conventional Blended Analysis shows more active learning

38. Efficiency: Modelling teacher time Learning activities for online courses Preparation time (fixed costs) • Guided TEL resources (model, AR) • Access to expositions (lecture videos) • Automated grading (MCQs, models) • Readings (pdfs) • Guided collaboration activities (wiki, AR) • Peer group discussion (forum) • Peer grading against criteria (share) • Tutored discussion (forum) • Tutor feedback (assignments) • Tutor-based assessment (exams) Support time (variable costs)

39. Modelling the teacher’s workload for increasing student cohort size Planning teacher time for a new online course at IOE

40. Modelling the teacher’s workload for increasing student cohort size From the Duke MOOC Total teacher time in hours Time for student support Time for preparation of resources Cohort size Scaling up will never improve thestudent support costs… unless…

41. Modelling the costs for increasing student cohort size … we develop some clever pedagogical patterns that support students at better than the 25:1 student:staffratio The question is – what are they, and how do we develop and share them? Invest in the teachers who can innovate with learning technologies!

42. Further details… tinyurl.com/ppcollector Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies (Routledge, 2002) (Chinese edition ECNU Press) Teaching as a Design Science: Building pedagogical patterns for learning and technology(Routledge, 2012) d.laurillard@ioe.ac.uk

43. Teaching as a design science: SUMMARY The global demand for HE requires investment in pedagogic innovation to deliver high quality at scale Technology-based pedagogic innovation must support students at a better than 25:1 student:staff ratio Teachers need the tools to design, test, gather the evidence of what works, model benefits and costs Teachers are the engine of innovation – designing, testing, sharing their best pedagogic ideas

44. The project partners The Learning Designer A TLRP-TEL project Oxford Liz Masterman (CoPI) Marion Manton (CoPI) Joanna Wild (RF) Birkbeck/LKL George Magooulas (CoPI) Patricia Charlton DionisisDimakopoulos IOE/LKL Brock Craft (RF) Diana Laurillard (PI) DejanLjubojevic (RF) LondonMet Tom Boyle (CoPI) RVC Kim Whittlestone (CoPI) Stephen May Carrie Roder (PhD Student) ALT SebSchmoller Rachel Harris LSE Steve Ryan (CoPI) Ed Whitley RoserPujadas (PhD Student) Project website at www.ldse.org.uk PPC at web.lkldev.ioe.ac.uk/PPC/live/ODC.html