Assessment Cycle or Circular File: Do Academic Librarians Use Information Literacy Assessment Data?

1. Assessment Cycle or Circular File:Do Academic Librarians Use Information Literacy Assessment Data? Megan Oakleaf & Lisa Hinchliffe Library Assessment Conference, August 2008

2. Methods - IL Assessment Data • Surveys • Tests • Performance assessments • Ex: evaluation of student bibliographies, research papers, or portfolios • Classroom Assessment Techniques (CAT) • Ex: muddiest points or minute paper • Etc. • Etc.

3. Barriers to Assessment From the literature: (Bresciani 2009) • Too little time/resources • Lack of knowledge or skills • Lack of process coordination • Lack of conceptual framework for assessment • Lack of collaboration with faculty • Lack of trust • Difficulties managing expectations

4. Research Questions • Do academic librarians use information literacy assessment data? • If so, how do they use the data? • If not, why not? • What can be done to encourage the use of information literacy assessment data in academic libraries?

5. Survey & Participant Details • 12-question survey using SurveyMonkey • Distributed to Information Literacy Instruction Listserv (ILI-L) and the ACRL Immersion Alumni listserv • 437 participants • 98% academic librarians • 96% conduct IL instruction • 35% supervise people who conduct IL instruction • 31% have been a librarian 1-5 years • 21% have been a librarian 6-10 years • 24% have been a librarian 20+ years

6. Do academic librarians assess information literacy skills? 76% (332) assess IL skills 24% (105) do not

7. Barriers to Assessing IL Skills

8. Too Little Time 26% I spend my time on other work responsibilities. 21% I’m unsure how to re-allocate my responsibilities to make time for assessment.

9. Too Few Resources 29% I don’t have the necessary staff. 19% Other library services and priorities monopolize available resources.

10. Lack of Knowledge & Skills 21% I have limited understanding of or expertise in assessment. 17% I’m concerned that I am not competent in conducting assessment. 15% I don’t know how to select & implement assessment tools. 14% I don’t know how to analyze assessment data.

11. Lack of Process Coordination 29% There is no centralized support/commitment for assessment. 22% There is no assessment coordinator to assist me. 18% Assessment is not valued in my library. 15% There is no assessment committee to assist me.

12. Lack of Conceptual Framework 18% It’s hard to integrate with college-wide assessment efforts. 17% It’s hard to grasp librarians’ role in assessing student learning.

13. Fear, Anxiety, Lack of Trust 17% I’m concerned that I am not competent in conducting assessment.

14. Lack of Collaboration with Faculty 21% Faculty involvement is limited. 19% Faculty consider the library an auxiliary service. 11% Faculty do not collaborate with librarians or vice versa.

15. Difficulty Managing Expectations 17% Many assessment tools don’t adequately measure information literacy.

16. To Assess IL Skills, Librarians Need: • More time • Help re-allocating their time • More staff • Greater understanding of IL assessment • Centralized support and/or a coordinator of assessment • Increased faculty involvement

17. Do academic librarians who assess IL skills have results from these assessments? 86% (276) have results 14% (46) do not

18. Barriers to Producing Assessment Results

19. Too Little Time 31% I spend my time on other work responsibilities besides producing assessment results. 24% I don’t have time for the added responsibility of producing assessment results. 21% I’m unsure how to re-allocate my responsibilities to make time for producing assessment results.

20. Too Few Resources 24% I don’t have the necessary staff to produce assessment results. 10% I won’t be rewarded for producing assessment results.

21. Lack of Knowledge & Skills 31% I have limited understanding of or expertise in producing assessment results. 14% I don’t know how to select & implement tools for producing assessment results.

22. Lack of Process Coordination 24% There is no centralized support/commitment for producing assessment results. 10% There is no assessment coordinator to assist me in producing assessment results. 10% There is no assessment committee to assist me in producing assessment results. 10% Assessment results are not valued on my campus.

23. Lack of Conceptual Framework for Assessment 17% It’s hard to integrate assessment results with college-wide assessment efforts.

24. Fear, Anxiety, Lack of Trust 10% I’m concerned that I am not competent in producing assessment results.

25. Lack of Collaboration with Faculty 24% Faculty do not collaborate with librarians or vice versa to produce assessment results. 17% Faculty involvement in producing assessment results is limited.

26. Difficulty Managing Expectations 31% Many assessment results don’t adequately measure information literacy. 17% Many assessment results don’t provide an adequate level of detail.

27. To Produce Results, Librarians Need: • More time • Help re-allocating their time or responsibilities • More staff • Greater understanding of how to produce assessment results • Centralized support for producing assessment results • Increased faculty collaboration in producing assessment results • Assessment results that adequately measure IL skills

28. Do academic librarians who have IL assessment results use them? 83% (228) use assessment results 17% (48) do not

29. Barriers to Using Assessment Results

30. Too Little Time 45% I spend my time on other work responsibilities besides using assessment results. 25% I don’t have time for the added responsibility of using assessment results. 18% I’m unsure how to re-allocate my responsibilities to make time for using assessment results.

31. Too Few Resources 15% I don’t have the necessary budget to use assessment results. 15% I don’t have the necessary professional development opportunities to use the assessment results. 15% I don’t have the necessary staff to use assessment results. 15% I won’t be rewarded for using assessment results. 15% Other library services and priorities monopolize available resources for using assessment results.

32. Lack of Knowledge & Skills 40% I have limited understanding of or expertise in using assessment results. 33% I don’t know how to select & implement tools for using assessment results.

33. Lack of Process Coordination 28% There is no centralized support/commitment for using assessment results. 15% The use of assessment results is not valued on my campus. 13% The use of assessment results is not valued in my library. 13% There is no assessment committee to assist me in using assessment results. 10% There is no assessment coordinator to assist me in using assessment results.

34. Lack of Conceptual Framework 35% It’s hard to grasp expectations for using assessment results. 33% It’s hard to integrate the use of assessment results with college-wide assessment efforts. 28% It’s hard to grasp librarians’ role in using assessment results. 13% It’s hard to grasp librarians’ role in using assessment results for accreditation purposes.

35. Fear, Anxiety, Lack of Trust 25% I’m concerned that I am not competent in using assessment results.

36. Lack of Collaboration with Faculty 30% Faculty do not collaborate with librarians or vice versa to use assessment results. 28% Faculty involvement in using assessment results is limited.

37. Difficulty Managing Expectations 48% The assessment results can’t be used because they don’t adequately measure information literacy. 40% The assessment results can’t be used because they don’t provide an adequate level of detail.

38. More time Greater understanding of how to use assessment results Training in how to use assessment results Training in how to select & implement tools for using assessment results Centralized support for using assessment results Clearer expectations for using assessment results Greater understanding of librarians’ role in using assessment results Increased faculty collaboration/involvement in using assessment results Methods for integrating assessment results with college-wide efforts Assessments results that adequately measure IL skills Assessment results that provide an adequate level of detail about IL skills To Use Results, Librarians Need:

39. What do academic librarians use IL assessment results for?

40. Ways Academic Librarians Use IL Assessment Results 96% To improve instruction 73% To increase student learning 58% To inform future assessment efforts 52% To provide data for accountability measures 50% To collaborate with faculty 36% To collaborate with other librarians 20% To argue for increased resources

41. Results Summary (n=437)

42. Closing the Loop…

43. So, what can be done to encourage academic librarians to assess IL, produce assessment results, & use the results to achieve the purposes of assessment?

44. From the literature: (Bresciani 2009) Educate Clarify Collaborate Coordinate Celebrate Be flexible Keep it simple What can be done to encourage collection & use of assessment data?

45. Educate Train librarians about • IL assessment in general, • tools for assessing IL skills adequately (and in detail), • tools for producing data, and • tools for facilitating the use of assessment data Inform library administration about • the time and • resources required to assess IL skills, as well as produce and use assessment results

46. Clarify Clarify the role of librarians in assessing information literacy as well as producing & using assessment data

47. Collaborate Collaborate with faculty, centralized campus assessment support processes and personnel. Participate in college-wide efforts to act on assessment results.

48. Coordinate Coordinate assessment efforts within the library by creating structures to support IL assessment.

49. Be Flexible Re-allocate job responsibilities of those tasked with assessment duties.

50. Questions? moakleaf@syr.edu ljanicke@illinois.edu