1 / 41

Weaving Assessment and Accountability into the Institutional Fabric

Weaving Assessment and Accountability into the Institutional Fabric. Victor M. H. Borden, Ph.D. Associate Vice President University Planning, Institutional Research, and Accountability (IU) Associate Professor of Psychology (IUPUI) Past President (AIR) Devoted Fan (M.O.U.S.E.)

hazel
Download Presentation

Weaving Assessment and Accountability into the Institutional Fabric

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Weaving Assessment and Accountability into the Institutional Fabric Victor M. H. Borden, Ph.D. Associate Vice President University Planning, Institutional Research, and Accountability (IU) Associate Professor of Psychology (IUPUI) Past President (AIR) Devoted Fan (M.O.U.S.E.) Former Spy (U.N.C.L.E) What if Everyone Actually Bought In? 2006 Arizona AIR Conference – March 31 – Phoenix

  2. Overview • What the H-E- is he talking about? • Is there some kind of a problem? • Bought into what? • What is it we do again and why? • What might we be doing and how?

  3. What’s Going On? • Is your workload picking up? • Is there less, if any down time? • Do you have time to clean up and decompress/reflect between projects? • Is there such a thing as “between projects” or just “among projects?”

  4. The Challenge • Increased demands for what we do • External demands • Improved literacy • Increased expectations as to what can be done • Technological advancements • Expectations for self-service • Our own effectiveness • No notable increase in resources

  5. What Exactly Do We Do? • What is IR? • Who uses IR? • For what?

  6. The Many Faces of IR

  7. Arizona Faces of IR

  8. As We See Ourselves?

  9. As Others See Us?

  10. What is IR? …research conducted within an institution of higher education to provide information which supports institutional planning, policy formation and decision making. • Saupe, 1990 – The functions of IR

  11. Brought Into What? • Using data, information, evidence, analysis to… • Plan – Figure out what to do • Implement – Figure out how to do it • Evaluate – Determine how well it is being done • Improve – Figure out how to do it better

  12. Who Uses It? • Senior administrators • Administrative directors • Administrative staff • Campus committees • Deans and department chairs • Faculty senates/councils • Faculty • Students and student groups • State, federal and commercial agency staff

  13. Who Could Use It? Every-friggin-body

  14. Enrollment management Resource allocation Management Program improvement Process improvement Planning and budgeting Accountability Climate assessment Student learning outcomes assessment Environmental scanning Economic impact Faculty salary equity Space utililization Grant development Marketing Knowledge management Compliance reporting Data administration and warehousing Student engagement Alumni engagement and cultivation Used for What HE Functions?

  15. What Can IR Be Used For? Every-friggin-thing

  16. What is it we Really Do? • Sit at our desks, producing tables and charts to hand off to those who requested them? • Facilitate organizational learning

  17. The IR Credo I realize that I will not succeed in answering all of your questions. Indeed, I will not answer any of them completely. The answers I provide will only serve to raise a whole new set of questions that lead to more problems, some of which you weren’t aware of in the first place. When my work is complete, you will be as confused as ever, but hopefully, you will be confused on a higher level and about more important things

  18. A New Definition of IR • The primary function of institutional research is to facilitate organizational learning for the continuous improvement of higher education institutions and systems.

  19. What’s important is not what the data say, It’s what we say and do about the data ~Victor Borden

  20. A Learning Paradigm • Planning, evaluation, and improvement all fit within a rational model • Learning incorporates uncertainty, ambiguity, and multiple styles • Individual learning and organizational learning are compatible concepts • Learning is mission critical for all educational institutions

  21. Norton & Kaplan’s BSC • Financial performance • Customer service and satisfaction • Process effectiveness and efficiency • Organizational learning

  22. Senge’s 5 Dimensions of Organizational Learning • Personal mastery • Mental models • Building shared vision • Team learning • Systems thinking

  23. Single- and Double-Loop Learning • Learning is the detection and correction of error (unintended consequences) • “Governing Variables” are those things what we feel are important to keep within acceptable limits • “Action Strategy” is what we do or plan to do to keep the governing variables within limits • “Consequences” are the intended and unintended outputs and outcomes • Intended: confirm our theory in use • Unintended: suggests error in our theory in use

  24. Single-Loop Learning • Governing variables not called into question • Adjustments made to action strategies at best • Defense mechanisms can readily arise to maintain single-loop learning Governing Variables Action Strategies Conse- quences

  25. Double-Loop Learning • Questioning the role of the framing and learning systems which underlie actual goals and strategies • Reflection is fundamental • Basic assumptions are confronted • Hypotheses publicly tested • Falsification is sought • Ego is laid aside Governing Variables Action Strategies Conse- quences

  26. Model I and II Org Learning • Single- and double-loop learning at the organizational level • Model I: Organizational members prescribe to a common theory in use • Organizational policies and practices inhibit change • Model II: Governing values, policies, and practices promote double-loop learning

  27. A Model I Learning Organization • Governing Variables • Tow the line • Win at all costs • Suppress negative feelings • Emphasize rationality • Action Strategies • Control environment and task unilaterally • Protect self and others unilaterally • Discourage inquiry • Consequences • Defensive relationships • Low freedom of choice • Reduced production of valid information • Little public testing of ideas

  28. A Model II Learning Organization • Governing Variables • Valid information is most important • Free and informed choice • Shared internal commitment • Action Strategies • Shared control • Participation in design and implementation of action • Consequences • Minimally defensive relationships • High freedom of choice • Public testing of ideas

  29. Domains of HEI Information Use • Two dimensions • Commonality of goals and objectives • Intentionality of information use • Four Quadrants • Model I • Model II • Disciplinary Scholarship • Nothing doing

  30. Individual/Private Each person works toward their own purposes Everyone works toward the same purposes Organizational/Shared Commonality of Goals and Objectives

  31. Intentionality of Evidence Use Subjective/biased Looks for and accepts evidence that supports a given conclusion Objective or balanced approach to identifying and examining evidence Objective/unbiased

  32. HEI Information Use Domains Organiz’l/shared Intentionality of Information Use  Commonality of Goals/Objectives  Individual/private

  33. Beyond Information Provider • It is one thing to give people the information they think they need to make decisions • It is quite another to engage our colleagues in using information to figure out how to improve our collective lot

  34. How Might we Expand Our Impact • Borrowing from colleague’s models • Structural arrangements • Brokering

  35. Colleague’s Models • Library • Collections; reference; information literacy; credibility assessment; online tools • IT • Centralized – Standards and supported platforms • Decentralized – Local Support Providers • Usability analysis • Stewardship • Professional Development (CTL) • Workshops; consulting; faculty fellows

  36. Other Useful Concepts • Outsourcing • Noel-Levitz; College Board • NSSE; CSRDE; Delaware • Structural arrangements • Split positions; hosted staff; exchanges • Contracted services

  37. Still Other Useful Concepts • Networking Structures • Coordinating committees • IR Council • Quality facilitators • Brokering • Linking to expert resources • Internal and external

  38. Summary • We are victims of our own success • We da man / We go, girl • The more we do to respond to demand, the more demand we create • We must go from providing fish to teaching people how to fish • We must learn together with our colleagues how to improve our fishing yield

  39. Implications • We need to expand our ideas about deploying IR products and services • More emphasis on coordination and collaboration • We need to develop ourselves as more informed and expansive higher education administrators • What doesn’t kill us will surely make us stronger

  40. Good to Great (Social Sector) Jim Collins

  41. What is IR? • What are we good at? • Figure out how to cull use evidence from the information droppings all around us • What are we passionate about? • Getting people to use evidence… • What do we have resources to support • By ourselves, not very much • With our colleagues, an H-E- of a lot more

More Related