Information Society Technologies Programme Accompanying Measures (IST-2001-32633)

1. Innovation in e-Learning Seminar Sponsor Information Society Technologies Programme Accompanying Measures (IST-2001-32633)

2. Innovation in e-Learning Seminar Objective To disseminate, stimulate discussion and creative suggestions with regard to identified best practices in e-Learning

3. Innovation in e-Learning Seminar Agenda 9:15 Introduction 9:30 Project Objectives and Applied Methodology 10:30 Break 10:45 Project Results – Case Studies 11:30 Roundtable Discussion 12:00 Lunch 13:15 Presentations and Input From Participants 15:15 Break 15:30 Next Steps 16:00 Closing Comments

4. Innovation in e-Learning Seminar Project Objectives & Applied Methodology

5. Project Objectives & Applied Methodology • Project Objectives • General Objective • Specific Objective • Applied Methodology • Identification of e-Learning Sites • Initial Evaluation • Structure • Applications • Final Evaluation • Learner Models • Soft Skills & Informal Learning • Communities of Learning • Dissemination Activities December 2002

6. Project Objectives General Objective To stimulate creativity, and to enhance education and training systems for lifelong learning. (IST 2001 Key Action 3.2.1 – Self Learning for Work) December 2002

7. Project Objectives Specific Objectives • To conduct a study on innovative structures and applications of e-Learning within the United States and Europe. • To foster innovative e-Learning development through dissemination of “best practice” innovative structures and applications. December 2002

8. Applied Methodology Identification of e-Learning Sites e-Learning Sites: • Internet based learning that is focused on the delivery of content (text, audio, video, animation, simulation) in a training format. Description Reasoning Reflection December 2002

9. Applied Methodology Identification of e-Learning Sites • Utilized the four leading search engines based on the Jupiter Media Matrix’s ratings for “audience reach”, 2002 (searchenginewatch.com). • Microsoft Network • Yahoo • Google • America Online Description Reasoning Reflection December 2002

10. Applied Methodology Identification of e-Learning Sites • Conducted a search with the term “e-learning” and selected only sites with a focus on students as the end user. • About 50 sites identified Description Reasoning Reflection December 2002

11. Applied Methodology Identification of e-Learning Sites • Search engines are the most common method for identifying Internet sites. Theme “Self-learning for work” - accessible existing sites. • Assumption: leading e-Learning sites have the knowledge to leverage search engine and directories to attract end users. • Assumption: the term “e-learning” is a universal term applied across Europe and the United States for Internet based learning. Description Reasoning Reflection December 2002

12. Applied Methodology Identification of e-Learning Sites • Restricted search due to assumptions – the term “e-learning” and e-learning sites ability to leverage search engines. • Other options are also highly restricted such as surveys of members of relevant associations (European Distance Education Network – EDEN) that rely on a significant response rate. There is always a low interest for participating in evaluations! Description Reasoning Reflection December 2002

13. Applied Methodology Identification of e-Learning Sites • Restricted search due to assumptions – the term “e-learning” and e-learning sites ability to leverage search engines. (Continued) • Possibly limited to English based sites – 6 % of the resulting sites were non English based. Description Reasoning Reflection December 2002

14. Applied Methodology Identification of e-Learning Sites • Resulting sites represented a diversity of organisation structures (private for profit, associations, etc.) but no traditional universities. • A lack of university focus on the general public market. • Opportunity to transfer innovative activities to the traditional university sector – dissemination activities. Description Reasoning Reflection December 2002

15. Applied Methodology Initial Evaluation • Identified a set of criteria to evaluate the sites’ structures and applications. • Evaluated the sites based on the set of criteria. • Identified the top 5 sites with regard to structures. • Identified the top 5 sites with regard to applications. • Identified the top 20 sites overall (structures / applications). Description Reasoning Reflection December 2002

16. Applied Methodology Initial Evaluation • Analysis of structures accounted for 7 specific criteria: • Objectiveness of the Mission – clearly stated/understood • Unique Design and Structure of the Interface • Aesthetics of the Web site – draw interest and appealing • Usability and Navigation – user friendly Description Reasoning Reflection December 2002

17. Applied Methodology Initial Evaluation • Analysis of structures accounted for 7 specific criteria (continued): • Adaptability to the Web – course content adapted or simply a written document • Appropriateness of the Course Structure – take into account subject (Accounting includes problem solving simulations) • Support Available – (contact information, call centre) Description Reasoning Reflection December 2002

18. Applied Methodology Initial Evaluation • Analysis of applications accounted for 4 specific criteria: • Technological Characteristics –course specific (Audio, video, animation, whiteboards) • Degreeof Interaction with OtherStudents (Synchronous interaction via audio/video/chat, email) • Degree of Interaction with a Tutor (Synchronous interaction via audio/video/chat, email) • Pedagogical Tools –course specific (exercises, quizzes, simulations) Description Reasoning Reflection December 2002

19. Applied Methodology Initial Evaluation • Previous works: • Previous studies within the e-learning field: LGUIDE – The e-Learning Experts • Thesis: “A Framework for Comparing Web-Based Learning Environments” Description Reasoning Reflection December 2002

20. Applied Methodology Initial Evaluation • Subjective component to the evaluation, although limited. • Short list of criteria sources – a need to review the sources utilized with other existing sources – American Society for Training & Development, Expert Papers • Clear need to develop widely accepted criteria to assess e-Learning practices – DG Education & Culture “eLearning call for Proposals”, September 2002 Description Reasoning Reflection December 2002

21. Applied Methodology - Recap December 2002

22. Applied Methodology Final Evaluation • Evaluated the top 20 e-Learning sites with regard to the following 3 areas: • Learner Models – focus on flexible, personal models for learning. • Soft Skills and Informal Learning – focus on the development of soft skills (communication, teamwork, leadership, project management, etc) and applying informal learning. • Communities of Learning – focus on identifying and implementing virtual communities, individuals with similar learning needs, within and across organisational boundaries. Description Reasoning Reflection December 2002

23. Applied Methodology Final Evaluation • Learner Models evaluation was based on 10 criteria: • Skills GapAnalysis – skill level evaluation prior to course • Tutor Availability • Student Flexibility – course scheduling, course navigation • Control of Screen Information – student level interface • Defined Objectives – course purpose, expected performance Description Reasoning Reflection December 2002

24. Applied Methodology Final Evaluation • Learner Models evaluation was based on 10 criteria (continued): • Self Assessment – periodically preferred • Content Integration– audio, video, slide show, animation • Document Management – bibliography, e-books • Help Facilities – beyond a tutor such as links, help desk • Simple User Interface – user friendly, consistent interface Description Reasoning Reflection December 2002

25. Applied Methodology Final Evaluation • Soft Skills and Informal Learning evaluation was based on 7 criteria: • Interaction Activity – promoted and actual interaction • Reward – recognition or reinforcement of accomplishments: diploma, certificate • Transfer – relate course to the student’s “real world” • Environment – positive environment, tolerance of errors, appropriate feedback Description Reasoning Reflection December 2002

26. Applied Methodology Final Evaluation • Soft Skills and Informal Learning evaluation was based on 7 criteria (continued): • Communication Interactivity – level of interaction: student / student and student / instructor • Motivation – ability to motivate the student, justify the need to learn • Student Centred – ability to manage depth and scope of learning, schedule, contribution. Description Reasoning Reflection December 2002

27. Applied Methodology Final Evaluation • Communities of Learning evaluation was based on 8 criteria: • Clearly Stated Purpose – establish a foundation for the community of learning • Experienced Moderators – ability to initiate and guide discussions • Registration System – to allow students to present themselves to the community and provide the community a structure Description Reasoning Reflection December 2002

28. Applied Methodology Final Evaluation • Communities of Learning evaluation was based on 8 criteria (continued): • Synchronous Tools – allow real-time interaction (chat rooms, discussion boards, virtual meetings) • Asynchronous Tools – ability to initiate and guide discussions (e-mail, message posting on web forums) • Group Problem Solving – foster groups with objectives within the course structure Description Reasoning Reflection December 2002

29. Applied Methodology Final Evaluation • Communities of Learning evaluation was based on 8 criteria (continued): • Scheduled Events – ability to increase student awareness of group activities on particular topics • Level of Interaction – final result: existence of actual communities with activity Description Reasoning Reflection December 2002

30. Applied Methodology Final Evaluation • EC defined key topics of learning • Previous works: • Previous studies within the e-learning field: LGUIDE – The e-Learning Experts • Thesis: “A Framework for Comparing Web-Based Learning Environments” • Various articles and reports from “experts” Description Reasoning Reflection December 2002

31. Applied Methodology Final Evaluation • Subjective component to the evaluation, although limited. • Short list of criteria sources – a need to review the sources utilized with other existing sources – American Society for Training & Development • Clear need to develop widely accepted criteria to assess e-Learning practices – DG Education & Culture “eLearning call for Proposals”, September 2002 Description Reasoning Reflection December 2002

32. Applied Methodology Dissemination Activities • Letter of Interest to about 750 individuals involved/interested in Distance Learning • About 60 responses directly interested in the project findings • Seminar • E-mail final reports • Website: spi.pt/innoelearning December 2002

33. Contacts Sociedade Portuguesa de Inovação Edf. Les Palaces Rua Júlio Dinis, 242 – Sala 208 4050-318 Porto, Portugal Tel: 351-22-6076400 Fax: 351-22-6099164 E-mail: markspinoglio@spi.pt www.spi.pt December 2002 Sociedade Portuguesa de Inovação