Robert McCormick Nai Li

1. Robert McCormickNai Li AN EVALUATION OF EUROPEAN LEARNING OBJECTS IN USE

2. AUTHORS’ INFORMATION • Robert McCormick; RITES, Faculty of education and language studies, The Open University, UK. • Nai Li; National Centre for Social Research, UK. Sinem UZEL- Fatoş BÖLÜKBAŞI

3. ABSTRACT • Most studies on reusable digital learning materials, learning objects (LOs), relate to their use in universities. • Few empirical studies exist to explore the impact of LOs on pedagogy, especially in schools. • This paper provides evidence from an evalution of use of LOs in schools. Sinem UZEL- Fatoş BÖLÜKBAŞI

4. ABSTRACT • The evidence is from a European Union-funded project, Context eLearning with Broadband Technologies (CELEBRATE), involving 500 schools in six countries across Europe, in a pilot to examine the impact of LOs on pedagogy. Sinem UZEL- Fatoş BÖLÜKBAŞI

5. INTRODUCTION • This article presents the analysis of both quantitative and qualitative studies of the views of teachers on LOs based on their experience of using the CELEBRATE LOs. • CELEBRATE, Context eLearning with Broadband Technologies, is an ICT project funded by the European Commission over 30 months from june 2002 until November 2004. Sinem UZEL- Fatoş BÖLÜKBAŞI

6. INTRODUCTION • The project started with the LOM definition to maximise the possibility of LOs being interoperable in compliant Virtual Learning Environments (VLEs). • According to this definition ‘Learning Objects are defined as any entity, digital or non digital that can be used or re-used or referenced during technology supported learning.’ Sinem UZEL- Fatoş BÖLÜKBAŞI

7. INTRODUCTION • Confling Interest Groups • CELEBRATE contained, within its various partners, all the interest groups that are found in the field: • Those whose concern is to produce LOs, whether they be commercial content providers or ministries of education and the like, who are making provision for schools. Sinem UZEL- Fatoş BÖLÜKBAŞI

8. INTRODUCTION • Those who are trying to make LOs available to users through various electronic systems, and have a concern for the technical issues. • Those who are concerned to bring ideas from contemporary views of learning, a largely academic and theoretically focused group. Sinem UZEL- Fatoş BÖLÜKBAŞI

9. RESEARCH DESIGN AND METHODOLOGY • The study used both quantiative and qualitive methods including routine data collceted on the Demonstration Portal, online questionnaire surveys, an interview study, experimental studies and classroom case studies. Sinem UZEL- Fatoş BÖLÜKBAŞI

10. RESEARCH DESIGN AND METHODOLOGY • In addition to a registration questionnaire to obtain background information on teachers, the study developed an online questionnaire to collect teacher views on, and experience of using CELEBRATE LOs. Sinem UZEL- Fatoş BÖLÜKBAŞI

11. RESEARCH DESIGN AND METHODOLOGY • More specifically the qustionnaire covered the following issues: • Experince of using LOs in general. • Reactions to the quality of design and content of CELEBRATE LOs. • Use of LOs in teaching and learning. • Issues of language and culture. Sinem UZEL- Fatoş BÖLÜKBAŞI

12. RESEARCH DESIGN AND METHODOLOGY • The questionnaire was originally constructed in English and piloted with project teachers in two workshops. • Then each pilot country project coordinator organised the translation of the revised version from English to their local language. Sinem UZEL- Fatoş BÖLÜKBAŞI

13. RESEARCH DESIGN AND METHODOLOGY • To complement the survey data from the qustionnaire, an interview study was developed. • It attempted to give some insight into teachers’ experience and perception of CELEBRATE LOs. Sinem UZEL- Fatoş BÖLÜKBAŞI

14. CELEBRATE TEACHER PROFILE • Teachers were invited to participate in the project and asked to register on the Demonstration Portal by completing the short registration questionnaire. • It aimed to capture information on teachers’ background such as age, gender and teaching experience and their experience of using ICT, to help build a profile of those involved in the CELEBRATE project. Sinem UZEL- Fatoş BÖLÜKBAŞI

15. CELEBRATE TEACHER PROFILE • A total of 770 teachers registered with an equal gender balance, from more than six countries, though they came mainly from Finland (42%) with representation from Hungary (14%), Israel (14%), Norway (13%) and France (12%). • There were very few teachers from the UK, only 4% of the total participants. Sinem UZEL- Fatoş BÖLÜKBAŞI

16. CELEBRATE TEACHER PROFILE • Teachers formed two groups with regard to ICT skills: high-ICT-skills group (41%) and core-ICT-skills group (59%). • The core-skills group reported being able to make use of a basic set of ICT skills, while the high-skills group also had a number of more complex capabilities. Sinem UZEL- Fatoş BÖLÜKBAŞI

17. CELEBRATE TEACHER PROFILE Sinem UZEL- Fatoş BÖLÜKBAŞI

18. OVERALL TEACHER REACTIONS TO THE LOS TEACHERS SAW AND USED • FINLAND: Teachers in Finland noted an improvement in students’ learning that they linked to the use of Los • Some students had improved their listening comprehension skills during the LO teaching period, which was discovered in the national listening comprehension examinations. • This involved students who do not watch television at their homes and hear English from there. Sinem UZEL- Fatoş BÖLÜKBAŞI

19. OVERALL TEACHER REACTIONS TO THE LOS TEACHERS SAW AND USED • They benefited from the LO teaching greatly. HUNGARİAN Teachers overall reaction to the use of LOs Hungarian teachers were even more positive, and this came out in both the questionnaire results and the interview study: • It is conspicuous by the analysis of the answers, that everybody’s opinion—without any exceptions!—was positive about the effect of the use of LOs. Sinem UZEL- Fatoş BÖLÜKBAŞI

20. OVERALL TEACHER REACTIONS TO THE LOS TEACHERS SAW AND USED • They think that, through using LOs, they can hold their students’ attention more successfully and they can persuade them easily into individual work and thinking. • It is also important that, according to teachers’ opinion, not only higher motivation level of students in and outside of the class could be observed, but also significant improvement of exam grades. Sinem UZEL- Fatoş BÖLÜKBAŞI

21. THE QUALITY OF THE DESIGN AND CONTENT OF LOS • Generally, teachers across the project were very positive about the CELEBRATE….LOs; almost 70% of the teachers believed that LOs were useful in teaching… Sinem UZEL- Fatoş BÖLÜKBAŞI

22. USABILITY AND FLEXIBILITY OF LOS • Overall, teachers in the project showed mixed reactions to the usability and flexibility. Here it is important to distinguish technical and pedagogic considerations. • For example,more than 56% of teachers reported that they encountered technical problems in using Los,and over a quarter of the total number of teachers felt that most LOs needed to be used with some special software such as Flash, Media Player etc. Sinem UZEL- Fatoş BÖLÜKBAŞI

23. EFFECTIVENESS OF LOS IN TEACHING AND LEARNING Over half of teachers (56%) believed that the use of LOs improved their teaching practices… According to the interviewed teachers, the LOs bring diversity in learning by presenting the topics to the students by means of multimedia, such as animation and sound effects.An LO can be more visual than a traditional textbook.

24. EFFECTIVENESS OF LOS IN TEACHING AND LEARNING However, 30% of the teachers had reservations about the ease of use of LOs for whole-class teaching, reporting that LOs did not fit into this type of teaching easily

25. CHARACTERISTICS OF LOS • The accounts in the literature on LOs see them as having several characteristics, namely granularity, reusability and that they should include pedagogy . • The evaluation results shed some empirical evidence on these issues and in this section we discuss them drawing more generally on the evaluation. • Granularity • Reusability • Pedagogy Sinem UZEL- Fatoş BÖLÜKBAŞI

26. Granularity • Granularity turned out to be less a function of size (of the ‘grains’), than of how integrated or modular LOs were. Integrated LOs tend to have a number of topics or ideas and be relatively complex, whereas modular ones focus on one idea or topic. • The evidence pointed to the need to take a modular approach. Sinem UZEL- Fatoş BÖLÜKBAŞI

27. Granularity • The first element of this evidence related to the definition of LO size in terms of curriculum topic focus. • The second element of evidence that supported the modular approach was that LOs proved difficult to match to teachers’ curricula. • When LOs were more integrated and larger, then it was likely that this match was more difficult to find. Sinem UZEL- Fatoş BÖLÜKBAŞI

28. Granularity • The third element is perhaps the most compelling, as it stems from the variety of uses to which LOs were put in terms of extending teachers’ repertoires in teaching. This has to be seen in terms of ‘blended learning’. • Thus, if LOs are going to be used in a blended approach in the classroom, this would prove more difficult if they were integrated together. Sinem UZEL- Fatoş BÖLÜKBAŞI

29. Granularity • The Fourth element of evidence, that of student autonomy. • If the student is to exercise choice of sequence, then having a pathway set out by an integrated series of LOs could be a problem (if it is at the extreme end of integration). Sinem UZEL- Fatoş BÖLÜKBAŞI

30. Reusability • There are three elements of reusability, namely interoperability, flexibility and modifiability. • Local technical problems (noted earlier) affected LO interoperability, and this has implications for minimum specifications for school systems to work across Europe. Sinem UZEL- Fatoş BÖLÜKBAŞI

31. Reusability • Flexibility of use relates to the degree of blended learning a teacher uses. Many LOs are designed for use in a purely electronic environment, perhaps within a VLE. • Where teachers use a mixture of normal classroom activities and LOs, then the flexibility relates to the tightness of the specification of the pedagogy of the LO, and the degree to which it can be adapted to fit a teacher’s pedagogy. Sinem UZEL- Fatoş BÖLÜKBAŞI

32. Pedagogy • The specification of the pedagogy of an LO is a controversial aspect of the characteristics of LOs and it assumes that the pedagogy resides within the LO rather than in the interaction of the way the teachers fit it into their own pedagogy and that of the LO activity. • The evaluation found that the classroom environment was where the control of the nature of the pedagogy resided, not the LO Sinem UZEL- Fatoş BÖLÜKBAŞI

33. CONCLUSIONS • The questionnaire survey and interview study evidence point to a generally positive reaction to the use of LOs by teachers in the six countries of Europe covered by the project. • This is a useful finding for such a large-scale project that started with little prior empirical evidence to guide the design and use of LOs in schools. Sinem UZEL- Fatoş BÖLÜKBAŞI

34. CONCLUSIONS • It is clear that teachers find them useful to support teaching and learning. However, one debate in the LO literature, that of having some element of pedagogy within the LO, is not supported. • The fact that teachers use LOs in variety of contrasting ways means that they are likely to be able to superimpose their own pedagogy on any LO, almost whatever the ‘designed’ pedagogy. Sinem UZEL- Fatoş BÖLÜKBAŞI

35. CONCLUSIONS • It appears that the granularity and interoperability characteristics are more significant in defining them as useful, particularly where they support resource-based learning. • Modular LOs that can run on any particular computer system is the basis of their attraction. Sinem UZEL- Fatoş BÖLÜKBAŞI

36. CONCLUSIONS • The match to the curriculum and the supply of sufficient LOs in national languages presents real problems for a European-wide repository of LOs. • There are no obvious answers to this outside a large-scale translation exercise combined with local adaptation of LOs. Sinem UZEL- Fatoş BÖLÜKBAŞI

37. CONCLUSIONS • In this paper we have not examined the data on teacher authoring of Los (and the project did not explore modification of content providers’ LOs by teachers), but there are indications that this might be a way forward. • This is particularly so given that teachers were generally unhappy about the fit to the curriculum, even when (in one country) the LOs were produced by the ministry. Sinem UZEL- Fatoş BÖLÜKBAŞI

38. THE CITE THIS ARTICLE • McCormick, Robert and Li, Nai (2006) ‘An Evaluation Of European Learning Objects In Use”, Learning, Media and Technology. • URL http://dx.doi.org/10.1080/1743988060089325 Sinem UZEL- Fatoş BÖLÜKBAŞI