building a resource for practical assessment adding value to value and impact n.
Skip this Video
Loading SlideShow in 5 Seconds..
Building a resource for practical assessment: adding value to value and impact PowerPoint Presentation
Download Presentation
Building a resource for practical assessment: adding value to value and impact

Loading in 2 Seconds...

play fullscreen
1 / 53

Building a resource for practical assessment: adding value to value and impact - PowerPoint PPT Presentation

  • Uploaded on

Building a resource for practical assessment: adding value to value and impact. Stephen Town University of York, UK Library Assessment Conference, Seattle Wednesday 6 th August, 2008. Summary. The SCONUL Value & Impact Measurement Program (“VAMP”) recap The Performance Portal

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Building a resource for practical assessment: adding value to value and impact' - Audrey

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
building a resource for practical assessment adding value to value and impact

Building a resource for practical assessment:adding value to value and impact

Stephen Town

University of York, UK

Library Assessment Conference, Seattle

Wednesday 6th August, 2008

  • The SCONUL Value & Impact Measurement Program (“VAMP”) recap
  • The Performance Portal
  • ‘Value’ options
    • UK drivers (TRAC)
    • Institutional Case: The Open University’s Best Value project
    • Benchmarking & national statistics
the university context from the 2006 library assessment conference after lombardi
The University Context (from the 2006 Library Assessment Conference, after Lombardi)

Universities have two “bottom lines”

  • Financial (as in business)
  • Academic, largely through reputation in
    • Research (the priority in “leading” Universities)
    • Teaching (& maybe Learning)
library pressures for accountability
Library Pressures for Accountability

The need is therefore to demonstrate the Library contribution in these two dimensions:

  • Financial, through “value for money” or related measures
  • Impact on research, teaching and learning

This also implies that “competitive” data will be highly valued

the aim role of universities their libraries cautions for measurement
The Aim & Role of Universities & their Libraries: cautions for measurement
  • Research, Teaching & Reductionism
    • ‘Mode 1’ Research & impact ‘transcendental’
    • ‘Mode 2’ Research & impact ‘instrumental’
    • Value, Price & ‘Mandarinisation’ of research and its support
    • Interdisciplinary research
    • Collaborative research across institutions
    • Learning as a set of discreet assessed modules
  • All of this may damage the idea of Libraries as ‘transcendent’, collective and connective services
sconul member survey findings
SCONUL Member Survey Findings
  • 70% undertaken value or impact measurement
  • Main rationales are advocacy, service improvement, comparison
  • Half used in-house methodologies; half used standard techniques
  • Main barrier is lack of tools,
    • Creating issues of time and buy-in
member survey conclusions
Member Survey Conclusions
  • There is a need to demonstrate value and that libraries make a difference
  • Measurement needs to show ‘real’ value
  • Need to link to University mission
  • Libraries are, and intend to be, ahead of the game
  • Impact may be difficult or impossible to measure
  • All respondents welcomed the programme, and the prospect of an available toolkit with robust and simple tools
vamp objectives
VAMP Objectives
  • New missing measurement instruments & frameworks
  • A full coherent framework for performance, improvement and innovation
  • Persuasive data for University Senior Managers, to prove value, impact, comparability, and worth
missing methods
Missing methods
  • An impact tool or tools, for both teaching & learning and research (from the LIRG/SCONUL initiative?)
  • A robust Value for Money/Economic Impact tool
  • Staff measures
  • Process & operational costing tools
program benefits
Program Benefits
  • Attainment & retention of Library institutional income
  • Proof of value and impact on education and research
  • Evidence of comparability with peer institutions
  • Justification of a continuing role for libraries and their staff
  • Meeting national costing requirements for separating spend on teaching and research
communities of practice
Communities of Practice

“groups of people who share a passion for something that they know how to do,and who interact regularly to learn how to do it better”

“coherence through mutual engagement”

Etienne Wenger, 1998 & 2002

vamp project structure
VAMP Project Structure
  • Analysis March-June 2006
  • Tools I (Impact ) - June 2007
  • Site Development - June 2007
  • Tools II (Value) - ?
  • CoP development
  • Maintenance

Community of



Member’s Forum



Home Page





Techniques in Use


the performance portal1
The ‘Performance Portal’
  • A Wiki of library performance measurement containing a number of ‘approaches’, each (hopefully) with:
    • A definition
    • A method or methods
    • Some experience of their use in libraries (or links to this)
    • The opportunity to discuss use
discussion tools
Discussion Tools
  • An experiment in social networking & Web 2.0 technologies
the ontology of performance
The Ontology of Performance
  • ‘Frameworks’
  • ‘Impact’
  • ‘Quality’
  • ‘Statistics’
  • ‘Value’
  • A visual Mind map?

European Framework for Quality Management (EFQM)


Key Performance Indicators

The Balanced Scorecard

Critical Success Factors

The Effective Academic Library


Impact tools


Detailed UK experience from LIRG/SCONUL Initiatives

Outcome based evaluation

Information Literacy measurement

More on research impact


Charter Mark

Customer Surveys


SCONUl Survey

Priority Research

Investors in People



Quality Assurance

ISO 9000s

‘Investors in People’ experience

Opinion meters

Quality Maturity Model


SCONUL Statistics & interactive service

HELMS statistics


Institutional experience of using SCONUL statistics for local advocacy


E-resource tools



Contingent valuation

‘Transparency’ costing

Staff & process costing, value & contribution

E-resource value tools

what is value
What is value?
  • Cost efficiency
  • Cost effectiveness
  • Cost comparison (Case 3)
  • Financial management process standards & audit
  • Financial allocation (Case 1)
  • Valuation
  • Value added
  • Return on investment
  • Best value (Case 2)
case 1 trac

Case 1: TRAC

UK Higher Education Transparency initiative 2000-09

transparent approach to costing
Transparent approach to costing
  • The standard method for costing in UK HEIs
  • Government requirement
  • Ending of cross-subsidy (T vs R)
  • Research funding based on full economic costing (fEC)
  • Positive effects on funding
  • Positive effect on pricing
  • All activity to be identified as ‘research’, ‘teaching’ or ‘other’
  • Library as other? or
  • All library activities either research or teaching, or a simplistic apportioning to each
  • Libraries omitted as a component of research costs, and therefore as a share recipient
ou best value program objectives
OU Best Value Program Objectives
  • To increase the business skills of library managers & staff
  • To develop skills to support customer-focused, cost-efficient management decision making
  • To develop benchmarking evaluation skills that balance quality, value and cost efficiency
  • Business reporting
  • Process costing and continuous improvement
  • Service planning
  • Benchmarking

‘to generate real accountability’

business reporting elements
Business reporting elements
  • Library business areas
  • Five PIs per area, including cost, quality & customer impact
  • Forecast, variance & remedial action

Has improved use of management information, efficiency, prioritisation and expenditure control

process costing
Process Costing
  • Complete process and stage costing
  • Average times and skill levels
  • Included enquiries, cataloguing, e-resources, IT support, document delivery, counter services

Has delivered justification for staffing levels against activity, staffing formulae, redeployment to priority areas, and process improvements

service plans
Service plans
  • Costed service plans to achieve medium term improvement and development through a rolling program

Included document delivery, enquiries, information literacy, and e-resources

program benefits and outcomes
Program benefits and outcomes
  • Staff development
    • cost-conscious decision-making
    • business skills
  • Management information improvement
  • Clarity about customers and use
  • Improved quality
  • Ability to ‘sell benefits’
international benchmarking initiatives
International Benchmarking initiatives
  • OU able to engage and lead an exercise against distance education Universities worldwide
  • In one 2008 international benchmarking study
    • Only one institution (out of eight) had a comprehensive costing model
financial statistical convergence
Financial Statistical Convergence
  • York Meeting, 2008
    • OCLC/RLG
    • ARL
    • SCONUL
    • CAUL
conclusion questions
Conclusion & Questions
  • What do mean by value?
  • Why do we not yet have a collective view on costing approaches?
    • Skills deficiency?
    • Lack of real need or real financial performance accountability?
    • We would rather not know?
    • Are we more intent on increasing budgets than seeking efficiency improvement?
  • The VAMP Subgroup of SCONUL WGPI

Maxine Melling, Philip Payne, Rupert Wood

  • The Cranfield VAMP Team, Darien Rossiter, Michael Davis, Selena Lock, Heather Regan
  • The Open University, Ann Davies
  • ‘Value’ Consultants, Sue Boorman, Larraine Cooper
  • Attendees at the York Statistics meeting