1 / 20

Institutional Research What problems are we trying to solve?

Institutional Research What problems are we trying to solve?. Mantz Yorke & Bernard Longden. Sorting out types of problems. “Reports that say that something hasn't happened are always interesting to me, because as we know,. there are known knowns ; there are things we know we know.

sydnee
Download Presentation

Institutional Research What problems are we trying to solve?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Institutional ResearchWhat problems are wetrying to solve? Mantz Yorke & Bernard Longden

  2. Sorting out types of problems “Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there areknown unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns the ones we don't know we don't know.” Donald H. Rumsfeld, Department of Defence - news briefing, February 12, 2002

  3. The challenge facing any University is to Be smarter and better manage theknown knowns; Inquire into university life to seek out the known unknownsand Reduce the risk of unknown unknowns asthey relate to: Demographic changes National policy drivers International opportunities

  4. Types of problem :Types of institutional research Two broad types of research Why is it going on type of research What’s going on type of research Descriptive Explanatory • Without theory, research is impossibly narrow. Without research, theory is mere armchair contemplation. (Silverman, 2001: 294)

  5. Institutional research - four faces Audience Summative & external Accountability Formative & internalfor improvement Information Presentation Administrative, Institutional Role Research forimprovement Educationalresearch Academic, Professional Volkwein (1999) has provided a useful analysis of the aims of IR – The four Faces of IR.

  6. Terenzini’s organisational intelligence Hierarchy of complexity Culture – knowledge of what a college is and where it has come from Major issues or decision areas facing the university Factual knowledgeanalytical skills

  7. What problems are we trying to solve?The view from inside - BL Audience Summative & external Accountability Formative & internalfor improvement Information Presentation Administrative, Institutional Role Volkwein (1999) has provided a useful analysis of the aims of IR – The four Faces of IR.

  8. Administrative/Formative-known knowns IR as Information Authority Administrative – formative and internal Emerging within institutions as: Information office Management information Planning and development Institutional Research Primary functions: Clarifying the known knowns Describing the university in data terms Educating the university community about itself

  9. What problems are we trying to solve? Information authority ‘Base data’ about the institution – profile and fact book Statutory returns Validating data – is this really us? Size and shape of the institution to assist in setting targets and plans Terenzini’s organisational intelligence level – Technical/analytical intelligence In the UK these are well developed possibly over load the university with data The challenge is to reduce the plethora of tables and cross tabs – to identify Key Questions that are crucial to the organisation for success

  10. What problems are we trying to solve? Presentation facility Marketing the best aspects – advocate role Audit and external accountability Performance measure League tables Terenzini’s organisational intelligence level – Technical analytical intelligence Primary functions: Conform data to agreed descriptors Performance Indicators Satisfy public funding legislative requirements Risks and concerns Spin and managing data Political pressure to maximise impact of data Denial about the message data gives

  11. Student Data flow Data informs funding- audit Student data Students Aggregate early return to HEFCE Data passed to HEFCEwho recreate Decemberaggregate return IndividualSummative activitydata return to HESA Butsame data also used to inform public through:Publications, commodification of data Rankings

  12. Data sources to assist with the questions UK • Higher Education Statistics Agency • Published data sets • HEDI • UCAS • Unistat • HEFCE (?) • National Student Survey US • National Centre for Education Statistics • IPEDS – universe institutional census • NPSAS – National Post Secondary Student aid Study • BPS – Beginning Post Secondary - Longitudinal Study • B&B – Baccalaureate & Beyond - Longitudinal Study • National Science Foundation

  13. Some propositions that we might need to address Do universities make good use of the data within their own university? Does IR constitute value for money? Is data critical to teachers, administrators, decision makers? Is the emphasis between data collection and analysis appropriate? Should the IR questions be articulated more clearly and data collected in response, or should we continue to collect data and then use what we have for policy decision? What lessons can be learned from the research community’s use of data sets in the US?’

  14. Comparison – sometimes interesting • UK HESA and US IPEDS – similar objectives and operation • US interest in scientific advancement not reflected in the UK • Possibly RAE provides nearest comparison • US positively encourages Institutional Research • Association for Institutional Research (US) in conjunction with National Centre for Education Statistics provides funded summer school to encourage analyses of public data • UK implicitly discourages Institutional Research • HESA charge for the use of our data!

  15. What problems are we trying to solve?The view from outside - MY Audience Summative & external Accountability Formative for improvement Role Research forimprovement Educationalresearch Academic, Professional

  16. Academic/Formative: known knowns and unknowns IR as Research for Improvement Academic – formative and internal Analysing (afresh?) existing data, or investigating institutionally-focused research questions, with a view to influencing practice. Problem: such activity is often scattered within an institution, and unco-ordinated.

  17. What problems are we trying to solve? Research for improvement Examples: Student feedback, including NSS Outcomes of quality assurance Assessment results, cross-tabulated with demographics, subject disciplines, etc. Incidence of plagiarism Research assessment exercise

  18. Academic/Summative: known unknowns IR as Educational Research Academic – summative and external, though with some formative and internal relevance Investigation of issues which are of wider relevance than to a single institution, but which may be triggered by an internal institutional need. Evidence needs to be particularly robust. Potential for dissemination of findings beyond the individual institution, in which case criteria for peer review may well apply.

  19. What problems are we trying to solve? Educational Research Examples: Pedagogical innovations of various kinds Assessment of achievement in workplace settings Variation in honours degree classifications across subject disciplines and institutional types Relation of PT employment to achievement on FT programmes Students’ experiences of HE (FYE; PT; HE in FE, etc) Non-completion

  20. In the end what answer can we give to Gnothi Sauton – ‘Know thyself’?

More Related