The Seventh International Blended Learning Conference University of Hertfordshire,13-14 June 2012

1. Evaluating Approaches to Developing Digital Literacy Skills “What do I need?” – evaluating learner perceptions of digital literacy skills development to inform enhancements to learning design Ingrid Nix and Marion Hall Faculty of Health & Social Care, Open University Kirsty Baker Open University Library The Seventh International Blended Learning Conference University of Hertfordshire,13-14 June 2012

2. Defined by European Commission as: ‘The confident and critical use of ICT for work, leisure, learning and communication.’ Digital literacy skills in graduates: Demanded by employers Required by UK HE Quality Assurance Agency Expected by learners: To effectively engage in technology-enhanced learning To be relevant to workplace (employability skills) But learners may: Not fully engage in skills development Prioritise subject-specific learning over skills Digital literacy

3. : Evaluating Approaches to Developing Digital Literacy Skills Project aims: • Explore learner perceptions of skills development • Develop understanding of factors motivating engagement with skills learning • Identify features of learning design that facilitate engagement and skills development

4. Outline of skills design approaches

5. Use of context/module-specific skills activities As used by SW1 and SW2

6. Use of HSC Resource Bank (HSCRB) generic skills activities YES NO AND/OR As used by HSC2

7. Data from 3 modules in Faculty of Health & Social Care: SW1 – Level 1 (first year) social work module SW2 – Level 2 (second year) social work module HSC2 – Level 2 health and social care module These use different approaches to skills development, based on: ‘Contextualised’ (context/module-specific) activities ‘Generic’ activities (re-usable across modules) Collected using mixed methods approach: Online reflective quizzes Quantitative + qualitative data Submission rate 23% (N=298) Interviews (N=18) qualitative data Data collection (Aug-Dec. 2011) PDF activities HSCRB

8. Volunteer interviewees: 9.5% (N=123) Randomly selected 6 from each module (total 18) Ensured both sexes represented where both volunteered SW1 no males volunteered for interview Interviewed face-to-face or by telephone Semi-structured interviews (1.5 hours) Audio recorded and transcribed All 3 authors acted as interviewer To standardise approach all authors did first interview together Each author participated in minimum 5 interviews (solo/pairs) Qualitative Data

9. Interview questions covered views/experiences & hypothetical questions about activity designs ICT (computer/software use) IL (finding, retrieving, evaluating the quality of information) ‘Burning issues’ question to enable participant to feed back concerns Interview transcripts were analysed using thematic analysis Interview Data

10. Transcripts coded using NVivo software Agreed the basis of a set of nodes (Bazeley 2007) Two transcripts each coded by 3 authors to increase reliability of coding Checked for interpretation, nodes adjusted Subsequent transcripts coded by one team member checked by another NVivo analysis

11. Results from 3 transcripts

12. Vicky (SW1, sponsored) Uses 2 computers at home to ‘run her entire life’ I very rarely use paper and pen nowadays, even shopping lists are typed, everything is e-mailed, spying on my children on the social networking sites. The main thing I use it for because I don’t use it for social media I think that’s tedious, I’m looking for information, a lot of information and buy a lot of books, and to me the internet is like the library of the world at your fingertips. Her boss has every IT gadget but her colleagues either love or hate it There have been so many IT system changes that colleagues can’t keep up They have all the skills a social worker needs. They can write a fantastic assessment and it falls down sometimes at not knowing which button to press so that the system will let you in. It appears that Vicky has confidence in using ICT and a critical perspective on its use Participants

13. Caroline (SW2, sponsored) Has wanted to be a social worker since 18 Currently Care manager, no guarantee of SW job on graduation Enthusiastic learner, has Masters in Policy and Professional Studies Enjoys OU technology-enhanced learning approach Didn’t enjoy red brick university degree Likes information at fingertips, no need to visit library in person Considers herself not very computer literate Fairly confident in ICT for home use (internet) But struggles e.g. locating files in folder structures Dubious about computers in workplace management pushes too much info via email, overloads staff coping mechanisms: colleagues avoid email, Caroline skim reads at work and in her OU studies Participants

14. Tracey (HSC2, self-funded) Secretary in a hospital, aiming for higher administrative post Already confident in ICT before module holds ECDL (European Computer Driving License) pursued independently during a previous job Considers her IL skills increased dramatically during module Someone close to her recently diagnosed with Crohn’s disease Been researching this online Considers that life can suddenly change & prevent study so her plans for Social sciences degree are tentative Participants

15. Motivation ICT confident: Not ICT confident: Caroline Vicky Tracey    ECDL • Selective use of activities • Develop new or re-usable skills • Desire time-saving tips • Tracey: Guidance needs to be • clear so can achieve • Printable (poor Broadband) • All activities undertaken • Marks/goals important • Desires • Own pace • Support • Clear rationale

16. Motivation ICT confident: Not ICT confident: Caroline Vicky Tracey    ECDL Caroline: It is about sometimes not understanding why you’re being asked to do things I think, but I’m that kind of person that will sit down and watch ‘what am I supposed to do on this’, and I will do things . . . I’m that kind of person I don’t always . . . some people just get on and do it but I do kind of challenge things until I can see why.

17. View of skills strategy Vicky Tracey Caroline ? Other students ok  ok  ?! Vicky: I didn’t really have a big issue with it. I was just grateful to get on the course really, so on a personal level it didn’t really matter to me. To do the degree if the ICT part was part of that, then so be it, I wasn’t having a moan, I’d just get on with it really.

18. View of skills strategy Vicky Tracey Caroline ? Other students ok  ok  ?! Caroline: I think I was aware of them but I don’t think I really read about them because it’s something that scared me, do you know what I mean, so I just thought ‘well we’ve got to do this, and that’s it’. So I was aware of them, yes.

19. View of skills strategy Vicky Tracey Caroline ? Other students ok  ok  ?! • Skills strategy clear • But inadvertently skipped by some students • Deliberately skipped guidance due to anxiety • Implication for designers: • To give due attention to provision of and signposting to skills strategy guidance • To do so regularly within programmes of study • To make it welcoming (not intimidating)

20. Identifying parts to complete Caroline Vicky Tracey ….Done  I can…  • Methodical approach • Fear of falling behind • Satisfaction all complete • V:welcome break, various techniques to identify new tips, titles/labels can be misleading (can make false assumptions know it all) so browsing is best • T: gauge if already has skill, • won’t waste time if familiar

21. V: To engage she values personal choice, to really want to learn the skill If not needed or can already do differently, will skip it T: Didn’t need all HSCRB activities to complete module tasks Integrated guidance fine, can skip past any extra detail & feel pleased already knows it Choice vs directed learning Integrated Methodical approach  Skipped HSCRB  Integrated also ok Personal choice I really want to learn • Integrated is helpful • Separate is too easy to skip • Fear of missing info • Doesn’t like ICT so wouldn’t seek out separate activities

22. Choice vs directed learning • Implication for designers: • Confident learners: • welcome optionality, can decide where to concentrate their efforts • Integrated activities do not present a problem as can skip past unnecessary guidance • Less confident: • welcome integration, clear directions & support • choice is dangerous (too easy to skip as lack motivation)

23. Indicators of content Descriptive titles and introductions Lists of learning outcomes, index lists, clickable menus At start of task ‘before’ and ‘after’ image is motivating (aptly conveys aim) Ability to scroll/browse Visual information Screen-captures to convey what expected outputs should look like To break up text Audio podcasts Useful for scenario setting, sense of ‘real world’ task Printable resources Useful reference documents for self and colleagues/family Text-based alternative (e.g. when poor broadband service) Boxed format of steps within task Clearly indicated steps in separate boxes Useful and engaging features

24. Visual information Provide more detailed examples to illustrate what outputs should look like Video screen-captures Illustrate processes, clicks and movements on screen Convey speed, efficiency, time-saving techniques Give added sense of support through audio giving instructions Indicators of content – quicker access to information Facility to hover/ rollover e.g. title to see outline of activity aims & contents Suggested enhancements

25. Use of skills in generic or subject-specific contexts Necessary X Tracey(Module activities) X Tracey(HSCRB activities) X Vicky Basic skills More complex skills X Caroline Subject specific Generic context Not necessary

26. Generic or subject-specific contexts • Generic contexts are suitable for HSCRB activity • If the skill only is in focus, then a generic context (outside the module) is fine e.g. holiday expenses (Vicky) But • Subject-specific context is preferred for Module skills activities • To maintain train of thought (Tracey) • Make meaningful, & interesting • Convey someone has taken trouble to make it relevant • The more complex the skill, if content/context is familiar, then you are able to deduce if it is correctly executed or not And • Detailed feedback only possible if everyone using same data (working through completed example in activity) • Disadvantage HSC2 approach: feedback not available if using module example/data • Advantage SW approach: Completed examples can help support outside one’s comfort zone (Caroline)

27. Balancing degree of challenge: new vs. familiar skill and context? Degree of challenge New C Context B A Familiar New 3 2 1 Skill

28. Tracey takes (risky?) selective approach even in assessed task Didn’t use formatting using Style headings Used existing manual method (“looks just the same”) lost marks, thought unfair Couldn’t see the rationale for learning the new skill which can facilitate automated tasks such as creating contents pages Implication for designers: Highlights the need to signpost Future usefulness of skills Pathway of future/progressive skills development Problematic experiences

29. More confident participants were able to make selective use of pathways offered Motivated to improve efficiency in own practices Desire speedy access to info from which to decide what to engage with Less confident preferred directed routes (less choice) to ensure all activities were completed Motivated by assessment/ personal learning goals Desires integrated activities, resources visible & accessible from one place Support features need to give sense of ‘human’ presence to build confidence Skills strategy guidance requires effective signposting Indicate future usefulness of skills to encourage investment in skills development Make engaging so not avoided e.g. by anxious learners Summary

30. Module-specific contexts preferred for module skills activities Generic is ok for basic skills where skill-only is in focus If module context is meaningful & familiar, then can deduce how to execute the skill Summary

31. Conclusions • Confidence level appears to be a factor in how learners’ approach engagement and what features best support them • Learners value clarity of purpose and benefit – this needs to be explicit in overarching guidance as well as within activity guidance • Learners value features which provide swift access to this information to enable them to make informed choices • Subject-specific context is perceived as enriching and supporting skills development • The degree of challenge for the learner will depend on the balance between new/familiar skill/subject-specific content

32. Next steps From analysing further transcripts to explore links between e.g.: • confidence levels/ previous educational qualifications and preference for context • qualitative and quantitative findings Consider the implications for learning designers of • Preferences for the use of subject-specific context

33. For funding and support the project team would like to thank: The OU Scholarship Fund The OU Faculty of Health & Social Care The OU Library From the OU Institute of Educational Technology Stephany Lay Robin Goodfellow Robert Farrow Acknowledgements Marion Hall Ingrid Nix Kirsty Baker

34. References • Bazeley, P. (2007) Qualitative data Analysis with NVivo, Second edition, Sage. • Braun V., Clarke, V. (2006) Using thematic analysis in psychology, Qualitative Research in Psychology, 3: 77-101. • Schegloff, E. A. (1997) “Narrative Analysis” thirty years later, Journal of Narrative and Life History, 7(1–4): 97–106.

35. More information See the EADDLES website: https://sites.google.com/site/eaddlsproject/home for: • More information about the project • Contact details for the project team