our most important challenge raising the assessment bar a challenge to our community n.
Skip this Video
Loading SlideShow in 5 Seconds..
Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community PowerPoint Presentation
Download Presentation
Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community

Loading in 2 Seconds...

play fullscreen
1 / 23

Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community - PowerPoint PPT Presentation

  • Uploaded on

Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community. Rick Luce, Emory University Library Assessment Conference Seattle – August 4, 2008. Environmental Scan. University mission evolving:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
our most important challenge raising the assessment bar a challenge to our community
Our Most Important ChallengeRaising the Assessment Bar: A Challenge to our Community

Rick Luce, Emory University

Library Assessment Conference

Seattle – August 4, 2008

environmental scan
Environmental Scan
  • University mission evolving:
    • Education reform and the Spellings commission  focus on outcomes and accountability
    • Globalization and competition
  • New research methods in a networked world:
    • Rise of eScience / eResearch = new ways to work, new needs and expectations
    • Data science & data scientists require new organizational environments
  • Social drivers:
    • Technology enabled social tools to connect & collaborate
my view after 2 years at emory
My View after 2 years at Emory

Aggressive Strategic Plan completed in first 4 months:

  • Strategic direction: (1) digital innovations, (2) special collections, (3) delivery of 21C access, resources, and services
  • $100M requested in new funds over 5 years
    • Concept approval for ~$36M for 3 new capital projects
  • Implement annual Business Plan – reviewed quarterly

None of which utilize current ARL statistics

we need a systems approach
We Need a Systems Approach
  • The human body is a system, our subsystems work together to keep us healthy
  • Research libraries are systems, requiring a management system to keep the subsystems working together to be healthy

Anyone who learns to see the organization as a system can never again feel satisfied with “improvement” initiatives which simply change staffing and the org chart but do not tackle the system itself

where assessment fits
Where Assessment Fits

Assessment – a method of planning for improvement

  • Catalyst for organizational change (not a quick fix)
    • Gain staff understanding for need for improvement & commitment to shared improvement goals

Ideally underpinned by a performance measurement matrix balancing:

    • Quality = customer defined goodness – internal & external
    • Time = speed, how fast is the response, agility
    • Cost = resources spent on people, processes, or organizational shifting or rework
performance measures our vital signs
Performance Measures: our Vital Signs
    • Statements without performance measures are wishful thinking -- without data, we don’t know
    • We all have volume or transaction data - provides no process insight
  • Move focus on product metrics to process metrics
      • Process performance (statistics, run charts, variation)
  • Getting to the ‘right’ metrics
    • What is the value equation?
    • How do we compare & differentiate ourselves?
    • Focus on : Customer, process, sponsorship metrics
hedgehog view constancy of purpose
Hedgehog View: Constancy of Purpose

What are we

best at?

What drives

our economic

(or value)


What are we



Adapted from: Jim Collins. Good to Great. 2001

id key success factors
ID Key Success Factors

Characteristics of successful organizations:*

  • Do something others cannot do
  • Do something well that others do poorly, or
  • Do something others have great difficulty doing well
  • ~10-15% (max.) of research libraries content / services are unique

What % of the budget resources support that?

*Prahalad and Hamel. The Core Competencies of the Corporation.

HBR, May/June 1990.

management principles of successful organizations
Management Principles of Successful Organizations
  • Strong customer focus
  • Effective leadership
  • Continuous improvement and learning
  • Management by fact
  • Fast response
  • Long-range view of the future
  • Results orientation
  • Cooperation, teamwork, partnering
a value proposition example lanl
A Value Proposition Example (LANL)
  • Scientific Productivity & Competitiveness

Enhancing research productivity and competitiveness

    • ROI = $4.50 : 1.00
  • Quality and Business Focused
    • 10 year average: 96% satisfaction rate; 25% delighted
    • Reduced cost per transaction by a factor of 16
    • Quality New Mexico ‘Roadrunner’ awards: 1997 & 2000
    • Federal Library of the Year 1999
    • “World’s Best Science Library” 2005 Blue Ribbon Review
follow the leaders
Follow the Leaders

Malcolm Baldrige National Quality Award: recognizes organizations practicing the most effective management methods

  • High performance is sustainable due to good management practices
  • Analysis of 600 winners over 10 years: growth = > 2.5 times as fast as peers, more than 2X more profitable
  • Examines approach, deployment and results
baldrige criteria for performance excellence
Baldrige Criteria for Performance Excellence
  • Leadership
  • Strategic Planning
  • Student, Stakeholder, and Market Focus
  • Measurement, Analysis, and Knowledge Management
  • Workforce Focus
  • Process Management
  • Results

Requires a system both in approach and deployment

baldrige applications lessons learned
Baldrige Applications: LessonsLearned
  • Accelerated learning using Baldrige framework
  • “System” is tough to integrate all at once

“Be patient, have discipline” Deming

  • Importance of supplier partnerships
  • Difficulty of language translation
  • Benchmarking data - time series data for competitors – couldn’t be obtained from libraries
  • Process for gaining and applying knowledge to improve business process performance from a study of current practices.
    • A means of using data to identify magnitudes and reasons for variances in performance.
  • Intent: comparative process data, best practices
    • Analyze the operation, know the competition & industry leaders
    • Incorporate the best of the best - become the new benchmark
missing from research library portfolios
Missing from Research Library Portfolios
  • Customer satisfaction index: (delight & loyalty) and perceived value
  • Product / service quality (defined by the customer)
  • Process and operational performance - cycle time, productivity
  • Employee satisfaction – learning, morale, training, alignment of strategy direction and rewards
  • Measuring supplier performance - quality, process variables, price competitiveness, overall ease of doing business
  • Financial: cost/value matrix, return on investment, cost avoidance
customer satisfaction metrics
Customer Satisfaction Metrics

Maturing our satisfaction assessment

Level 1 – Satisfaction surveys: ‘happiness meters’

Level 2 - What’s important  analysis of customer importance & satisfaction levels

Level 3 - How do we rate against best in industry

See it from the customers eyes

measures that matter
Measures that matter

Align library vital signs with the organization’s drivers

  • Quality of product and processes
  • Innovation
  • Research leadership
  • Brand identity
  • Growing market share
  • Reducing new product development time
  • Ability to attract and retain employees
  • Credibility
avoiding pitfalls
Avoiding Pitfalls
  • Measures that don’t focus on strategy
  • No accountability
  • Too many initiatives
  • Forgetting larger organizational drivers
  • Lack of discipline
  • Insulating researchers and managers from scholarly communication issues

No action without a plan, no plan without data

business scorecard desired business results
Business Scorecard - Desired Business Results

1. Customer focus:

  • Satisfaction, loyalty, value-added

2. Product quality:

  • E.g., accessibility, usability, accuracy, completeness

3. Operational process performance:

  • Productivity, competitiveness, cycle time

4.High performance workforce:

  • Organization performance assessment, formal process changes, employee satisfaction

5. Prestigious reputation -- output results

  • Strategic performance results, benchmark results, external assessment scores
any road will do if the destination is unknown
Any road will do if the destination is unknown

The journey to truly superior performance is neither for the faint of heart nor for the impatient.

The development of genuine expertise requires struggle, sacrifice, and honest, often painful self-assessment.

HBR: Anders Ericsson (FSU), Michael Prietula (Emory), Edward Cokely (Max Planck)


If you wish to do something for the community, build a road. If you wish to do something better for the community, build a bridge – Chinese Proverb

Together let’s build the bridge to a new

level of assessment practice,

supporting continuous improvement

and focusing on outcomes and impact.