r
Download
1 / 38

RAE & REF - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

r esearch. HEFCE. RAE & REF. Presented at Council of Deans, Seminar, Manchester, Feb 2009. By Hugh McKenna Karen Cox. BRIEF BACKGROUND. Conducted jointly by the Higher Education Funding Councils for England (HEFCE), Scotland (SHEFC), Wales (HEFCW) and Northern Ireland (DEL)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'RAE & REF' - nell


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Academic business clinical research innovation facility abc rif

research

HEFCE

RAE & REF

Presented at

Council of Deans, Seminar, Manchester,

Feb 2009

By

Hugh McKenna

Karen Cox


Brief background
BRIEF BACKGROUND

  • Conducted jointly by the Higher Education Funding Councils for England (HEFCE), Scotland (SHEFC), Wales (HEFCW) and Northern Ireland (DEL)

  • Purpose: to produce ratings of research quality for the HE funding bodies to determine research funding allocations to HEIs


Rae 2008
RAE 2008

  • Principles of Equity, Diversity, Equality, Expert Review, Clarity, Consistency, Continuity, Credibility, Efficiency, Neutrality, Transparency.

  • All types of research, including practice-based research, applied research, basic/strategic research, interdisciplinary research.

  • Assessment based on peer review.

  • 15 Panels and 67 Sub Panels


Sub panel 11
Sub Panel 11

  • Chair

  • Prof Hugh McKenna, University of Ulster

  • Panel Secretaries

  • Mrs Lilian Caras, Brenda Purkiss

  • Members

  • Ms Sarah Buckland, INVOLVE

  • Prof Charlotte Clarke, University of Northumbria

  • Prof Karen Cox University of Nottingham

  • Prof Dame Pauline Fielding,

  • Prof Martin Johnson, University of Salford

  • Prof William Lauder, University of Dundee

  • Prof Karen Luker, University of Manchester

  • Prof Dame Jill Macleod-Clarke, University of Southampton

  • Dr Maggs Maguire, Department of Nursing, Scottish Executive

  • Professor Dave Thompson, University of Leicester

  • Prof Ruth Northway University of Glamorgan.

  • Prof Anne Marie Rafferty, Kings College London

  • Dr Kate Seers, University of Warwick

  • Prof Roger Watson, University of Sheffield

  • Prof Anne Williams, University of Wales, Cardiff


Sub panel 12
Sub Panel 12

  • Chair

  • Prof Julius Sim, Keele University

  • Panel Secretaries

  • Mrs Lilian Caras, Brenda Purkiss

  • Members

  • David Billington, Liverpool John Moores

  • Michael Campbell, University of Sheffield

  • Aedin Cassidy, University of East Anglia

  • Stephen Downes, University of Ulster

  • Pamela Enderby, University of Sheffield

  • Bernard Gilmartin, Aston University

  • Martin Griffin, Aston University

  • Jenny Hewison, University of Leeds

  • Tracey Howe, Glasgow Caledonian University

  • Ian Kitchen, University of Surrey

  • Di Newham, King's College London

  • Margaret Nicol, Queen Margaret University Edinburgh

  • David Rogers, University of Portsmouth

  • Diane Scutt, University of Liverpool

  • Debbie Sell, Great Ormond Street Hospital

  • Christina Victor, University of Reading

  • Keith Wafford, Eli Lilly and Company


Main panel c
Main Panel C

  • Professor Maggie Pearson (Chair)

  • Professor David Williams (10)

  • Professor Hugh McKenna (11)

  • Professor Julius Sim (12)

  • Professor Robert Hider (13)

  • Professor Mi Ja Kim

  • Professor John Stamm

  • John Stageman

  • MRC/AMRC/ESRC/DoH/Scot Gov Observers

  • Secretariat


Rae 20081
RAE 2008

  • January 2005 Guidance to panels issued;

  • Summer 2005 Guidance on submissions issued;

  • Autumn 2005 Consultation on criteria and working methods;

  • February 2006 Final criteria and working methods issued;

  • 31 July 2007 End of assessment period for research income and research student data

  • 31 December 2007 End of publication period (cut-off point for publication of research outputs)

  • 31 October 2007 Census date

  • 30 November 2007 Closing date for submissions

  • December 2008 Results published


Definition of research
DEFINITION OF RESEARCH

  • Research - original investigation undertaken in order to gain knowledge and understanding. It includes work of direct relevance to the needs of commerce and industry, and to the public and voluntary sectors; scholarship*; the invention and generation of ideas... etc, where these lead to new or substantially improved materials, devices, products and processes...

    [* The creation, development and maintenance of the intellectual infrastructure of subjects and disciplines, in forms such as dictionaries, scholarly editions, catalogues and contributions to major research databases.]

  • It excludes routine testing and routine analysis of materials, components and processes (such as for the maintenance of national standards…).

  • It also excludes the development of teaching materials that do not embody original research”.

  • It excludes Audit


Criteria working methods sub panel 11
CRITERIA & WORKING METHODSSub Panel 11

  • “Research activity relevant to the disciplines of nursing and midwifery, specialist community public health nursing and all the contexts within which they operate, including policy, practice, education and management”.

  • Proportional weighting given to quality of Research Outputs, Research Environment and Research Esteem;

  • Opportunity to refer to other Sub Panels and specialist advisors;


Criteria working methods sub panel 12
CRITERIA & WORKING METHODSSub Panel 12

  • “The UOA includes (but is not limited to): biomedical sciences; nutrition and dietetics; optometry and orthoptics; radiography; podiatry; occupational therapy; physiotherapy; speech and language therapy; arts therapies; health promotion; psychosocial and ethical aspects of health and healthcare; associated health services research (to include methodological work on quantitative or qualitative procedures)”;

  • Proportional weighting given to quality of Research Outputs, Research Environment and Research Esteem;

  • Opportunity to refer to other Sub Panels and specialist advisors.



Academic business clinical research innovation facility abc rif

Overall

Overall

Overall

Overall

Overall

The overall quality profile

The overall quality profile

The overall quality profile

The overall quality profile

quality profile

quality profile

quality profile

quality profile

quality profile

comprises the aggregate

comprises the aggregate

comprises the aggregate

comprises the aggregate

of the weighted profiles

of the weighted profiles

of the weighted profiles

of the weighted profiles

produced for outputs,

produced for outputs,

produced for outputs,

produced for research outputs,

4*

4*

4*

4*

4*

4*

3*

3*

3*

3*

3*

2*

2*

2*

2*

2*

1*

1*

1*

1*

1*

u/c

u/c

u/c

u/c

u/c

research environment and

research environment and

research environment and

research environment and

Quality level

Quality Level

Quality Level

Quality Level

.

.

.

.

esteem indicators

esteem indicators

esteem indicators

esteem indicators

% of research

% of Research

% of Research

% of Research

20

20

20

20

20

20

25

25

25

25

25

30

30

30

30

30

15

15

15

15

15

15

10

10

10

10

10

10

activity

Activity

Activity

Activity

Research environment

Research environment

Research environment

Research environment

Research environment

Research environment

Esteem indicators

Esteem indicators

Esteem indicators

Esteem indicators

Esteem indicators

Research Esteem

Research outputs

Research outputs

Research outputs

Research outputs

Research outputs

Research outputs

4*

4*

4*

4*

4*

4*

4*

4*

4*

3*

3*

3*

3*

3*

3*

3*

3*

3*

2*

2*

2*

2*

2*

2*

2*

2*

2*

1*

1*

1*

1*

1*

1*

1*

1*

1*

u/c

u/c

u/c

u/c

u/c

u/c

u/c

u/c

u/c

4*

4*

4*

4*

4*

4*

4*

4*

3*

3*

3*

3*

3*

3*

3*

3*

2*

2*

2*

2*

2*

2*

2*

2*

1*

1*

1*

1*

1*

1*

1*

1*

u/c

u/c

u/c

u/c

u/c

u/c

u/c

u/c

4*

4*

4*

4*

4*

3*

3*

3*

3*

3*

2*

2*

2*

2*

2*

1*

1*

1*

1*

1*

u/c

u/c

u/c

u/c

u/c

25

25

25

25

25

25

25

25

25

25

25

25

25

25

25

25

25

25

15

15

15

15

15

15

15

15

15

20

20

20

20

20

20

20

20

20

15

15

15

15

15

15

15

15

15

30

30

30

30

30

30

30

30

20

20

20

20

20

20

20

20

10

10

10

10

10

10

10

10

20

20

20

20

20

20

20

20

20

20

20

20

20

20

20

20

15

15

15

15

15

25

25

25

25

25

35

35

35

35

35

15

15

15

15

15

10

10

10

10

10

eg

eg

eg

eg

70%

70%

70%

70%

70%

eg

eg

eg

eg

20%

20%

20%

20%

eg

eg

eg

eg

10%

10%

10%

10%

(Minimum 5%)

(Minimum 5%)

(Minimum 5%)

(Minimum 5%)

(Minimum 5%)

(Minimum 5%)

(Minimum 5%)

(Minimum 5%)

(Minimum 50%)

(Minimum 50%)

(Minimum 50%)

(Minimum 50%)

How did this work in reality?

The overall quality profile

comprises the aggregate

of the weighted profiles

produced for research outputs,

4*

3*

3*

2*

2*

1*

1*

u/c

u/c

research environment and

Quality level

.

esteem indicators

% of research

20

25

25

30

30

15

10

activity

4*

4*

3*

3*

2*

2*

1*

1*

u/c

u/c

15

15

25

25

35

35

15

15

10

10

25%

5%



Overview sub panel 11
Overview - Sub Panel 11

  • 761 individuals were returned, submitting 2,851 outputs, mostly from peer reviewed journals;

  • Selectivity in number of staff returned;

  • Selectivity in type of staff returned;

  • Selectivity of panel/sub panel;

  • 2001 – 43 submissions, 2008 - 35 submissions;

  • All thirty-five submissions were reviewed in detail by every sub-panel member;

  • All quality profiles represent the consensus of the sub-panel as a whole;

  • Strict cognisance was taken of equal opportunity issues and individual staff circumstances.


Rae 2008 submissions to sub panel 11 the strengths
RAE 2008: Submissions to Sub Panel 11- THE STRENGTHS

  • 44th out of the 67 subject areas in the RAE;

  • Internationally recognised research in all submissions;

  • Strengths in translational/applied research;

  • Good service user involvement;

  • Positive impact on patient/client care;

  • Good interdisciplinary work – various disciplines;

  • Good PhD completions in review period (446);

  • Good international collaborations;

  • Good methodological and theoretical sophistication;

  • Strategic research appointments;

  • Research linked to national and international priorities;


Rae 2008 submissions to sub panel 11 the strengths1
RAE 2008: Submissions to Sub Panel 11- THE STRENGTHS

  • Impact on national and international health care policy;

  • Substantial competitive research grant capture (> £103 M);

  • Modern and high quality infrastructure;

  • Robust staffing policies;

  • Emphasis on capacity building and sustainability;

  • Realistic and sustainable research strategies;

  • Highly focused, strategically sound, financially viable;

  • Good links between NHS, Voluntary bodies and HEIs;

  • Esteem indicators - numerous awards, involvement in prestigious national and international committees, funding bodies & editorial boards.


Rae 2008 submissions to sub panel 11 the weaknesses
RAE 2008: Submissions to Sub Panel 11- THE WEAKNESSES

  • In some institutions service user/public engagement was very limited;

  • Few departments were actively engaged in technology and knowledge transfer activities;

  • Very few links with industry;

  • Some researchers working in isolation in poorly developed research environments;

  • Lone researchers and small disparate groups working on the same topic area;

  • Little evidence of activity around the economic evaluation of what nurses and midwives do or on the evaluation and impact of new roles;


Rae 2008 submissions to sub panel 11 the weaknesses1
RAE 2008: Submissions to Sub Panel 11- THE WEAKNESSES

  • Fewer outputs on educational and managerial issues;

  • Research into aspects of care fundamental to nursing and midwifery were not sufficiently evident in submissions;

  • Over one third of the 1,283 research studentships were supported by the submitting HEIs themselves;

  • The doctoral degrees awarded per research student FTE were lower than expected;

  • Only fifty-nine research-active staff were returned as designated ‘early career researchers’;

  • Insufficient critical mass, inadequate onward investment and inferior succession planning.


Overview sub panel 12
Overview- Sub Panel 12

  • 68 submissions - 18 to this sub-panel for the first time;

  • 1,622 individuals returned, submitting 6,216 outputs;

  • External research income totalled £355.7 million;

  • Varying levels of selectivity exercised by HEIs, in submissions and outputs;

  • Heterogeneous in substantive and methodological terms;

  • Very wide distribution of quality across the submissions;

  • All quality profiles represent the consensus of the sub-panel as a whole;

  • Strict cognisance was taken of equal opportunity issues and individual staff circumstances;


Rae 2008 submissions to sub panel 12 the strengths
RAE 2008: Submissions to Sub Panel 12- THE STRENGTHS

  • Clearly-articulated and focused research strategy;

  • Coherent and synergistic research groupings, with an appropriate ratio of researchers to research foci;

  • Strong evidence of sustainability;

  • Well-balanced portfolio of research funding, from competitive peer-review funding sources;

  • Appropriate and productive links with partners in the NHS and industry, and with service users;

  • Good evidence of translational research.;

  • Outputs showed rigorous and innovative methodology;

  • Research outputs had clear evidence of actual or potential impact on practice or policy.


Rae 2008 submissions to sub panel 12 the strengths1
RAE 2008: Submissions to Sub Panel 12- THE STRENGTHS

  • Evidence of internationally excellent or world-leading research in virtually all the disciplines – especially biomedical science, nutrition and optometry;

  • Evidence of investigations into the efficacy or effectiveness of therapeutic interventions;

  • Large body of important work on the underpinning basic sciences;

  • Studies providing valuable insights in the social and psychological aspects of health, illness, and the delivery of professional care;

  • Appropriate and well developed collaborative research links

  • Some Departments were leading important international research collaborations

  • 50% increase over review period in PhDs awarded (2,096).


Rae 2008 submissions to sub panel 12 the weaknesses
RAE 2008: Submissions to Sub Panel 12- THE WEAKNESSES

  • Submissions were heterogeneous, consisting of a number of small and seemingly unrelated areas of work, raising questions as to cohesiveness and future proofing;

  • Lack of clear evidence of sustainability;

  • Lack of strategic focus, often with disparate research groupings;

  • Underdeveloped research infrastructure;

  • Inappropriately large number of research foci for the number of staff submitted;

  • Weak funding profile;

  • Discrepancy between the profiles for outputs and environment;


Rae 2008 submissions to sub panel 12 the weaknesses1
RAE 2008: Submissions to Sub Panel 12- THE WEAKNESSES

  • A number of outputs did not meet the RAE research definition;

  • Need for an increased focus on the many areas within AHPs where the evidence base is as yet underdeveloped

  • 3,426 new research studentships - institutional self-funding (36%); Research Councils (11%);

  • High quality doctoral work may not have attracted, or been unable to access, Research Council funding;

  • Research outputs that were of poor methodological quality and/or of little potential impact;

  • Poorly developed interdisciplinary and/or institutional collaboration.


Purpose of qr points from hefce
Purpose of QR(points from HEFCE)

Creating a sustainable and flexible national baseline capacity which enables the sector to respond strategically to a changing external environment

To maintain a research base of world leading quality across the full range of disciplines

To encourage institutions to take investment decisions which lead to excellence in research, funded selectively by reference to robust indicators of research quality.

Invest in innovative research, including in new fields and opening new lines of enquiry, supporting early-career staff who aspire to grant funding

create potential to make connections across subjects


Ref current state of play
REF –current state of play

HEFCE suggests reduce the burden on institutions

Will be designed to take better account of the impact research makes on the economy and society.

A pilot exercise for bibliometric indicators of excellence is now underway

HEFCE will set out its plans for all aspects of assessment, including user-focused research and subjects where bibliometrics are not appropriate, by summer 2009.


Framework for assessment
Framework for assessment

RAE assesses a body of research activity in terms of:

  • Quality of outputs

  • Research environment

  • Esteem

  • HEFCE suggests indicators in REF may also capture

  • How a research environment is supportive of User valued research

  • Evidence that specific work is esteemed for its quality by particular audiences


Tools and approaches for assessment
Tools and approaches for assessment

HEFCE suggests assessment is likely to include:

  • Bibliometric analysis

    • Citations?

  • Expert review of outputs

  • Other available statistical indicators

    • Students (PG) and completions

    • Income

    • Studentships


Cunning plans
Cunning Plans

  • Succession Planning;

  • Increase and support Early Career Researchers (ECR)s;

  • Ruthless selectivity in all things;

    • Staff appointments and support for ECRs -Home grow Vs recruit;

    • Publish for high citations;

    • Research Grants from prestigious bodies – NIH guidelines;

    • Network and collaborate for best esteem nationally and Internationally;

    • NHS-HEI collaborations;

    • Impact – does the research make a difference?

  • Service user involvement from design to dissemination;

  • Interdisciplinarity in the widest sense; esteem is important – who you work with

  • Translational & policy Research;

  • Internal Vs External Studentships;

  • Address policy issues;

  • Increase the amount and quality of pedagogic research.


Conclusion
Conclusion

  • The sub-panel were impressed with most of the submissions and with the pattern of 3 and 4 star research;

  • There are many models of good practice from which developing research groupings can learn in terms of research activities, outputs, environment and esteem;

  • Investment by Governments, funding bodies and universities has increased research capacity and developed research leaders;

  • These funding streams need to be sustained and enhanced if the upward trajectory and momentum are to continue and if the quality differentials between the strongest and weakest departments are to be addressed.


Academic business clinical research innovation facility abc rif


Alison tierney scottish exec
Alison Tierney, Scottish Exec meeting of Academy of Nursing, Midwifery and Health Visiting Research (UK) 26/02/2009.

  • Need a continuous and stable funding stream to maintain and build on progress

  • In Scotland, over 40% submitted to other panels, e.g. social policy, psychology – some may be best researchers

  • What do we do with low quality and sub-Rae researchers?

  • Approx 10% N&M HEI staff submitted to Rae

  • How do we build structures and processes that best support HEI/Health Services partnerships in research and how best to measure outcomes?

  • Few joint academic-services authors of papers (7-10%?)


Developing hei nhs partnerships for research
Developing HEI/NHS partnerships for research meeting of Academy of Nursing, Midwifery and Health Visiting Research (UK) 26/02/2009.

  • Sustainable partnerships

  • Joint targets for research

  • Capacity building jointly agreed

  • Jointly managed research programmes

  • More shared infrastructure (as in medicine)

  • Joint and Honorary appointments at senior level

  • Clinical-academic careers (see Post Registration Career Framework for Nurses, Welsh Assembly consultation document, November 2008)


Professor maggie pearson chair rae 2008 main panel c
Professor Maggie Pearson meeting of Academy of Nursing, Midwifery and Health Visiting Research (UK) 26/02/2009.Chair, RAe 2008 Main Panel C

  • 70 HEIs in nurse education

  • 1996 36 HEIs and 396.5 staff submitted

  • 2001 43 HEIs and 578.4 staff

  • 2008 36 HEIs and 641.7 staff

  • So, over half did not submit – does this matter?


Academic business clinical research innovation facility abc rif

  • 1996 11.1 staff per submission meeting of Academy of Nursing, Midwifery and Health Visiting Research (UK) 26/02/2009.

  • 2001 13.5

  • 2008 17.8

  • 7 HEIs submitted only in 1996

  • 5 HEIs submitted only in 2001

  • 14 ‘dropped out’

  • 9 ‘new entrants’ in 2008

  • 12 HEIs submitted to all 3 exercises

  • 7 high quality


2008 c report

Stronger groups meeting of Academy of Nursing, Midwifery and Health Visiting Research (UK) 26/02/2009.

Strategic, strong leadership, clear foci

Peer reviewed, competitive funding

Critical mass (no lone researchers)

Institutional investment

National/international collaborations

Weaker groups

Lack of strategy

Weak leadership

Small, disparate group

Lone researchers

2008 C Report