Assessment in Academic Libraries 高校图书馆评估 工作

1. Assessment in Academic Libraries高校图书馆评估工作 Heidi Card ULS Librarian, Assistant to the Director on Research & Special Projects University of Pittsburgh hrc5@pitt.edu

2. What is Assessment?什么是评估？ • To determine the importance, size, or value • 确定被评估对象的重要性、规模或价值 • To evaluate • 对被评估对象给出评价

3. Assessment Activity in Libraries图书馆评估活动 • Biggest jump in assessment activities occurred between 1990-2004 • 1990-2004年间，图书馆评估活动发生了质的飞跃 • Impetus was service driven and user centered • 推动评估发展的动力是：服务驱动和以用户为中心 • 91% in ARL survey started assessment activities to learn more about their customers • 美国研究图书馆协会的调查表明，91%的高校均启动了评估活动，以此来更多地了解图书馆的用户

4. More Reasons to Assess更多的评估理由 • Improve student learning outcomes (instruction) • 提高学生的学习效果（培训） • Foster a learning organization for library faculty and staff • 为图书馆的员工创建学习型组织 • Proving value of library’s role as it relates to university goals • 证明图书馆存在的价值，因为图书馆与大学的建设目标是紧密相连的

5. Typical Assessment Practices 典型的评估方式 • Suggestion Box • 意见箱 • User survey • 用户调查 • Gathering statistics • 收集统计数据 • Web usability testing • 网站可用性测试 • Focus groups (staff & patrons) • 关注不同类型群体（工作人员与读者）

6. Areas of Library being Assessed图书馆经常被评估的领域 • Library web site • 图书馆主页 • Electronic resources • 电子资源 • User instruction • 用户培训 • Reference and collections • 参考咨询和馆藏建设 • *Administrative functions (human resources, marketing, financial services) are least evaluated – 30% of survey respondents have not assessed these areas. • 行政职能（例如：人力资源、图书馆营销和经费管理等）是最少被评估的内容-30%的调查反馈者没有涉及到这些方面的评估。

7. Who Does Assessment?谁来对图书馆进行评估 • Coordinator • 管理协调人员 • Committee • 委员会 • Assessment department • 专业评估部门

8. Successful Assessment Plan成功的评估计划（1） • A focus on supporting the patron’s information and communication needs • 关注对读者信息需求和交流需求的支持 • Performance measures that are included in the strategic plan • 实施方案纳入了图书馆发展规划 • A commitment to supporting assessment from the Libraries’ leadership • 获得图书馆领导的委托和支持 • Staff recognition of the value of assessment • 员工认可评估的价值

9. Successful Assessment Plan成功的评估计划（2） • Staff support for and participation in assessment initiatives • 员工支持并主动参与评估活动 • Continuous communication with patrons • 与读者不断的交流与互动 • Routine collection and analysis of data and user feedback • 定期收集、分析数据和用户反馈 • Commitment to applying assessment results in decision-making • 提交评估结果给上级部门应用于决策参考

10. Assessment Coordinator/Librarian评估协调人员/馆员 • Analyze, interpret, report on assessment activities • 分析、解释和报告评估活动 • Consult with staff on assessment methods and needs • 就评估的方法和需求与图书馆工作人员进行磋商 • Perform assessment activities • 组织并实施评估活动 • Coordinate/collaborate with other institutionalunits • 与其他相关单位进行协调/合作

11. Examples of Assessment Positions/Plans评估岗位/计划实例1 • Assessment Librarian, University of Texas Southwestern Medical Center评估馆员，得克萨斯大学西南医学中心 • Assessment Librarian position created in September 2006, reporting to the Assistant Vice-President for Library Services. 2006年9月设置了评估馆员的岗位，并向负责图书馆对外服务工作的副馆长助理报告工作 • Assessment plan drafted by the Assessment Librarian and approved in July 2008 评估馆员起草了评估计划，并于2008年7月获得通过 • Assessment plan includes details about assessment and evaluation activities through FY 2009-2010, including the project's rationale, assessment/evaluation method, population studied, etc. 2009和2010两个财政年的评估计划包括一些关于评估活动的细节，如：评估方案的原则、评估的方法和人群研究等等

12. Examples of Assessment Positions/Plans评估岗位/计划实例2 • Director of Service Innovations and Resource Planning, Cornell University服务创新和资源规划主任，康奈尔大学 • The mission of the Research and Assessment Unit (RAU) was revised in 2007. The Unit has a staff of 2.60 FTE. The Director reports to the Senior Associate Librarian for Public Services and Assessment. 2007年变更了研究和评估部门的任务。该部门有2.6个全日制员工。部门主管向负责图书馆公共服务和评估工作的副馆长报告工作 • RAU takes needs-driven, adaptive approaches. It creates 6-month milestones and has established workflows for each assessmentacitvity they are engaged in. 研究和评估部门采用需求驱动和适应性方法。它设置了一个6个月评估进程表，并为参与的每一个评估活动制定工作流程

13. Examples of Assessment Positions/Plans评估岗位/计划实例3 • Assessment & Marketing Librarian, Columbia University • 评估及营销推广馆员，哥伦比亚大学 • Assessment & Marketing Librarian position created in 2006, reporting to the Director of Access Services. Commits 50% of time to assessment. 哥大的评估及营销推广馆员的岗位设立于2006年，该岗位向对外服务的负责人汇报工作。这个岗位50%的工作时间用于评估工作 • Assessment plan was created by a representative, ad-hoc group whose research included literature reviews, interviews with key staff, and review of recent assessment data. 评估计划由特设小组代表制定，该小组的研究任务包括文献评价、与重要岗位工作人员交流和对近期的评估数据进行检查 • The plan is tied to the Libraries' 3-year planning cycle and provides both a conceptual framework and identifies assessment priorities. 该计划与图书馆三年规划周期相结合，既提供了一个概念框架，同时也确定评估的优先级别

14. Examples of Assessment Positions/Plans评估岗位/计划实例4 • Assessment Director, University of Chicago评估主任，芝加哥大学 • Assessment Director position established in 2007, reporting to the Head of Access Services and Assessment, manages one 15-hour/week student assistant. 2007年设立评估主任岗位，向负责对外服务和评估工作的主管领导报告工作，该岗位有一位每周工作15小时的学生工协助 • Plan drafted by Assessment Director and reviewed by Head of Access Services and Assessment. Approved by Library Director July 2008. 评估主任起草评估计划 ，提交给负责对外服务和评估工作的主管领导进行审定。2008年7月图书馆馆长通过了该计划 • Plan includes a detailed action plan for the year and is implemented by Assessment Director with the help of 5-member Assessment Project Team. Team convenes September 2008. 计划包括了评估工作的年度实施方案，该方案由评估主任带领一个5人评估项目组执行。评估项目组成立于2008年9月

16. Assessment Websites评估网站 • Often disseminated via website/page, often as a “data-bank” with reports and statistics • 通常通过网站/主页发布评估工作的内容，网站或主页往往也充当评估报告和统计数据的“数据资料库” • On-going projects and reports • 正在进行的项目和报告 • Milestones • 进程表 • Generally for internal use • 评估的结果一般供内部使用

17. Sample Library Assessment Sites图书馆评估网站样例 • Yale University • 耶鲁大学 • University of Washington • 华盛顿大学 • California State University • 加州州立大学 • University of Kansas • 堪萨斯大学

19. ARL Sponsored Library Assessment Conference 美国研究图书馆协会举办的图书馆评估会议

20. In 2007, the University of Pittsburgh developed a suite of Web accessible online information literacy tutorials using Adobe Captivate. Because of a lack of a compatible learning management component for Captivate at the University, the technical team developed an alternative to creating appropriate quizzes to test students on their comprehension and acquire feedback on the tutorials. 2007年，匹兹堡大学利用Adobe Captivate软件开发了一套基于互联网的在线信息素养教程软件，由于Adobe Captivate开发工具里缺少高校学习管理功能，匹大软件技术团队便又开发出了一种可供选择的创建适合于提问和测试的学习系统，以检验学生的理解能力和获取他们对学习教程的反馈意见 Using Survey Software to Assess the Effectiveness of Information Literacy OnlineTutorials 利用调查软件评估信息素养在线教程的效果 John Fudrow & Patricia Duck ULS Poster in ARL 2008 Assessment Conference2008年美国研究图书馆协会评估会议中有关匹兹堡大学图书馆评估工作内容的资料介绍

21. ULS Poster in ARL 2008 Assessment Conference2008年美国研究图书馆协会评估会议中有关匹兹堡大学图书馆评估工作内容的资料介绍 • Consequently, the team used an open source survey software, Lime, to create both the information literacy quizzes and obtain feedback on the tutorials. This presentation will present the pros and cons of using survey software as an assessment tool for information literacy. • 因此，该技术团队运用开放源码调查软件、Lime语言创建了信息素养检测和教程意见反馈的系统。本ppt里将会说明运用调查软件作为信息素养能力评估工具的利弊

22. Association of Research Libraries:美国研究图书馆协会：StatsQUAL 统计调查工具 • ARL Statistics 美国研究图书馆协会统计表 • LibQUAL+ 基于网络的图书馆服务质量评估工具 • ClimateQUAL 组织氛围与多样性调查 • DigiQUAL 基于网络的图书馆数字资源评估工具 • MINES for Libraries 图书馆信息挖掘工具

23. Assessment @ ULS

24. Collections 馆藏 User surveys 用户调查 Focus groups 专题小组 Outside consultant (Special Collections) 馆外查阅者（特藏） Services 服务 Statistics (gate counts, reference)统计（到馆人次，参考咨询等） Web usability testing 网上使用量测评 Focus groups 焦点群体 Information Literacy 信息素养 SAILS 信息素养技能标准化评估 IL and Assessment Working Group信息素养和评估工作组 Assessment Librarian评估馆员 Current Assessment Tools现行常用的评价工具

25. Transforming our libraries to reflect a culture of assessment is essential to increasing our success with customers and stakeholders, and maintaining relevancy in a competitive environment. 转变图书馆的运行模式以反映一种评估文化是为了增加我们与用户和股东合作的成功概率，以及在竞争相对比较激烈的环境中始终保持不败的关键 -- Lakos, A. and Phipps, S. (2004). Creating a Culture of Assessment: A Catalyst for Organizational Change, Libraries and the Academy, 4 (3), pp. 345-361. The Johns Hopkins University Press, Baltimore, MD. -- Lalkos，A.和Phipps，S.(2004).创造一种评估文化：组织机构 变化的催化 剂，图书馆与高等学校，4（3），P345-361.约翰 霍普金斯大学出版社，巴尔的摩，马里兰洲.

26. Assessment Plan评估计划 • Assessment Librarian hired in 2007 to develop assessment plan • 2007年聘用评估图书馆员是为了进一步完善评估计划 • Member of IL/Assessment Working Group • 信息素养成员 / 评估工作组 • New strategic plan = Culture of Assessment • 新的战略计划 = 评估文化 • Developing department goals • 发展中的部门目标

27. ULS StatisticsULS的统计表 • Web Traffic: Net Insight • 网站流量：网络洞察力 • Pittcat 匹兹堡大学图书馆书目系统 • DRL 数字化研究图书馆 • Ejournals 电子期刊 • D-Scribe 数字化书写 • Databases: Serial Solutions • 数据库：系列解决方案

28. ULS “Data Bank” PlansULS“数据资料库”计划 • previous assessment data will be archived and accessed via a dashboard interface • 以前的评估数据将被存档，并通过数字键盘来访问 • entry of new data will be automated or updated manually • 通过自动或手动输入方法更新评估数据 • used by administration and the statistics group • 提供行政管理部门和统计组使用

29. Information Literacy and Assessment Working Group, 2006信息素养和评估工作组，2006 • charged with developing an information literacy assessment program for the ULS, building on prior bibliographic instruction and assessment experience • 负责为ULS开发信息素养评估程序，优先建立馆藏书目培训指南，积累评估经验 • 3 subgroups (Instruction, Technical, and Ex-Officio) • 下设3个小组（教学组、技术组、委员组-依职权成立的小组） • created online tutorials (and revising existing ones) • 创建在线教程（修订现有教程） • finding new ways to promote informationliteracy, assisting with assessment and integration of information literacy into the curriculum (through the use of the SAILS test and discipline-based rubrics) • 寻找新方法推动信息素养教育，在教学课程中融入信息素养（通过使用SAILS测试软件和基于学科评分标准的方式） • offering informational workshops for ULS colleagues • 为ULS的同事举办信息研讨会

30. Began as a Kent State University initiative to develop an instrument for standardized assessment of information literacy skills • 源于肯特州立大学最初开发的信息素养技能标准化评估的工具 • Test is a web-based tool that allows libraries to document information literacy skill levels for groups of students and to pinpoint areas for improvement • 测试使用一种基于网络的工具，该工具可以记录学生信息素养技能的水平，并能精确定位学生需要改进和完善的地方 • Test questions are at varying difficulty levels to measure information literacy • 试题可以设置不同的难度等级以测评出学生的信息素养水平

31. How Can Libraries Use SAILS Data?图书馆如何使用SAILS数据？ • Use it to identify specific IL gaps of students • 运用数据识别学生们具体的信息素养能力的差异 • Demonstrate to departments the specific IL needs of their students and partner to address • 展示不同院系的学生所需要的特定的信息素养需求 • Eventually use this base data as a means of measuring the impact of IL instruction • 最后，还可以将这些基本数据作为测评信息素养教学效果的方法

32. Assessment of ULS Librarians匹兹堡大学图书馆系统的馆员评价 • Review process • 评议程序 • Peer Review Committee (PRC) • 同行评议委员会（PRC） • 9 faculty librarians9名专业图书馆员 • ClimateQUAL (not used by ULS) 组织氛围与多样性调查（尚未被匹大图书馆使用）

33. Culture of Assessment评估文化 • Planning & Budget Report • 计划与经费预算报告 • Strategic Plan 战略规划 • Department Goals部门目标 • Based on measurable objectives 基于可测评的目标 • Reflecting strategic plan 与整体战略规划相一致 • Individual evaluations • 个体评价

35. 匹兹堡大学图书馆系统（ULS）计划与经费预算报告（FY10）匹兹堡大学图书馆系统（ULS）计划与经费预算报告（FY10）

36. Any Questions?问题？

37. 谢谢！

38. References参考文献 • ARL SPEC Kit 303, 2007, • 学术期刊图书馆全文数据库存贮程序教育计算机303，2007， • Assessment Plan, Columbia University Libraries (2007-2009). Retrieved September 14, 2009, from https://www1.columbia.edu/sec/cu/libraries/bookmarks/img/assets/9436/CUL_Assessment_Plan.pdf • 评估计划，哥伦比亚大学图书馆（2007-2009）. • https://www1.columbia.edu/sec/cu/libraries/bookmarks/img/assets/9436/CUL_Assessment_Plan.pdf检索与2009年9月14日 • Hernon, P. & Dugan, R. E. (2009). Assessment and evaluation: What do the terms really mean? C&RL News, 70(3), 146-149. • Hernon.p和Dugan,R.E.(2009).评价和评估：这两个术语的真正概念是什么？C和RL评论，70（3），146-149. • Lakos, A. and Phipps, Shelley. (2004). Creating a culture of assessment: A catalyst for organizational change. Portal-Libraries and the Academy, 4(3), pp. 345-361. • Lakos，A和Phipps，Shelley.(2004).创造一种评估文化：组织机构变化的催化剂.图书馆入门和高等学校，4（3），P345-361.