Tracking Context and Attention Metadata for Multilingual Technology Enhanced Learning

1. Tracking context withusage and attention metadata in multilingual Technology Enhanced Learning Riina Vuorikari European Schoolnet / Open Univ. of the Netherlands Bettina Berendt KU Leuven www.berendt.de

2. „Learning Object Repositories foster re-use and improve learning“ • Really? • Ochoa (2008), Vuorikari & Koper (2009): re-use ~ 20%

3. A basic challenge: context Researching for a term paper in ... Biology Political Science

4. Motivation(application goals) • Use case • Technology Enhanced Learning / Learning Object Repositories • teachers from different linguistic and country backgrounds • Goal: foster re-use of resources • Assumption: understanding user context can improve system behaviour • Approach: • consider UAM as part of context measurement, specifically: • Investigate different roles of social tagging • Profit from parallel research in different areas

5. Motivation (general - workshop) • To exploit usage and attention metadata: need to • define what properties of usage to use and why • how to gather data on that property • what to do with that information

6. Motivation(general – our work) • Automatically gathered indicators of context, attention, ... • What‘s common across fields? • How can we communicate better? • What‘s specific in T.E. learning? • How can tags help learning & teaching? • How to bridge (e.g. lingual) barriers? • What are the best research methods for answering all these questions?

7. Types of context in TEL

8. What is context? • “Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.” (Dey, 2001) • Any information not explicitly carried by the “surface level“ of an interaction with a computational system • Surface level: • user issues a query, • user accesses a resource, ... • We will distinguish between • Macro-context • Micro-context

9. Macro-context • educational level • formal and informal learning • delivery setting (distance, blended, ...) • intended users and user roles • ...

10. Micro-context • User • trait • state • Interaction • atomic vs. activity structure • implicit vs. explicit interest indicators • Background knowledge / „semantic enrichment“; often w.r.t. material (Berendt, 2007)

11. Tags and context

12. Background knowledge What‘s in a tag? item user • material • user • interaction • „writing“ • „reading“ tag

13. Tags as interest indicators (1) – or: measuring and using interaction-context • Comparative search efficiency: • Search option [A|B|...]  a target LO • Click-through rate aka confidence (fraction of successful searches) • Path length • In LOR: Search with ... • explicit search (for resources w/o tags or ratings): 4.4 searches / resource • community browsing (tagclouds, lists, pivot browsing): 3.9 searches / resources • social information (community browsing or explict search with interest indicators (tags, ratings)): 2.8 searches / resource  Recommendation of search options?!

14. Tags provide context to better utilize background knowledge • Can tags be used for enriching existing metadata of educational resources in a multilingual context? • 30% of tag applications matched with descriptors from a multilingual thesaurus, which had also been used to index these learning resources (Vuorikari et al., 2009)  “Thesaurus tags” • Thesaurus tags provide for • adding properties to tags (e.g., relation to a concept in a multilingual thesaurus, language) • linking resources to multilingual thesaurus descriptors  can support retrieval of resources in a multilingual context • Cf. convergence and quality tendencies in tagging in general (Bollen & Halpin, 2009; Hayes et al., 2007)

15. What is user context – the case of language • User models have many different variables • Language (first language, further languages, proficiency, preferences): an important descriptor of users • Language situation (interaction in a first or second language): an important descriptor  Linguistic trait and state context variables!

16. Measures / indicators of user language • self-profiling that explicitly addresses the question (“your mother tongue?”) • IP address • browser settings • language of the currently used interface • language of search terms and tags • known or inferred language of tagged resources • NB: indicator of the language situation: compare user language with interface/resource language (e.g., Berendt & Kralisch, 2009)

17. Tags as interest indicators (2) – or: tags for measuring (re-)use • bookmarking and personal collections of digital learning resources as a proxy for the use and re-use of resources • When coupled with user and resource location / language: • become proxy for (re-)use across national and language boundaries (e.g., Vuorikari & Koper, 2009)

18. Why support trans-lingual re-use? (1): Situation on the Web • What would be an „ideal“ language situation on the Web? • One H0: ~ amount of materials proportional to number of first-language speakers of a language • Not the case – non-English languages are severely under-represented on the Web due to • Resource-creation behaviour • Link-setting behaviour • Link-following behaviour • Attitudes towards resources in different languages (Berendt & Kralisch, 2009)

19. Why support trans-lingual re-use? (2): User preferences • Theoretical reasons (cognitive effort), supported by implicit and explicit user preferences: • Clear tendency to access materials in one‘s mother tongue in a medical information system • 17% of users of a LOR had saved only content in their native language in their Favourites • LORs: pragmatic reasons (curricula) • Search-engine give better results when queried with linguistically correct (rather than Latinized) spellings • But: individual differences depending on proficiency in English! (Berendt & Kralisch, 2009; Vuorikari & Koper, 2009; Blanco & Lioma, 2009)

20. Using context for improving trans-lingual re-use: How? (1) • better-organised search result lists • first a ranked list in the preferred language, • then a ranked list in a second language, etc. • done: Google for general search results and by LeMill (lemill.net) for learning resources. • organise tag clouds by language or country as in LRE (lreforschools.eun.iorg)

21. Using context for improving trans-lingual re-use: How? (2) • for users with weaker language preferences; for „unknown“ users; ...“: • Make exceptions to the foregoing by recommendations using • travel-well tags: • terms with the same or similar spelling in most languages • ex.: technical terms like “mathematics”, place and person names • How to identify them? • via multi-lingual thesaurus or similar • tag has been assigned by users from different languages • Tag has been assigned to resources in different languages • travel-well resources: • Have been bookmarked by people from different languages, country contexts, … (Vuorikari & Ochoa, 2009)

22. Using context for improving trans-lingual re-use: How? (3) • Direct recommendations 1 (see above): • Recommend content or tags in a preferred language • Indirect recommendations: • E.g., recommend bookmark lists of other users with a similar “language preference profile” • Profile similarity: “extensional” and/or “intensional”: degree of tolerance for mixed-language resources, results, etc. • Direct recommendations 2: • repositories could specifically encourage users who are competent in “smaller” languages also to author content in their language.

23. Outlook

24. Some questions ... and now yours! (Thx!) • How to best couple „content“ and „architecture“ research • How to bring together research from Web analytics / e-Commerce, search-engine research, digital libraries, TEL, ... • How to complement the exploratory research with experimental research effectively and efficiently • How to best combine different methods (validate that the dependent variables used mean something!) • Which privacy/security questions arise? • What is specific for TEL?

25. References • Most references can be found in the paper, available at www.cs.kuleuven.be/~berendt/Papers/TEL_context_attention_vuorikari_berendt.pdf • In addition: • Bollen, D., & Halpin, H. (2009). An Experimental Analysis of Suggestions in Collaborative Tagging. In Proc. Of WI-IAT’09, Milan, Italy, 15-18 Sept 2009. IEEE Computer Society Press. • Hayes, C. et al. (2007). ... In Berendt, B., Hotho, A., Mladeni\v{c}, D., & Semeraro, G. (Eds.) (2007). From Web to Social Web: Discovering and deploying user and content profiles. Workshop on Web Mining, WebMine 2006, Berlin, Germany, September 18, 2006, Revised Selected and Invited Papers. LNAI 4737. Berlin etc.: Springer.