The rae is dead
1 / 15

The RAE is Dead! - PowerPoint PPT Presentation

  • Uploaded on

The RAE is Dead!. Long live the REF!. The Changes. The RAE looks at three main areas: Outputs Environment Esteem We are used evaluations of Environment and Esteem being “ informed ” by metrics The intention is that under the REF all three areas will be metric determined .

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'The RAE is Dead!' - Lucy

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
The rae is dead l.jpg

The RAE is Dead!

Long live the REF!

The changes l.jpg
The Changes.

  • The RAE looks at three main areas:

    • Outputs

    • Environment

    • Esteem

  • We are used evaluations of Environment and Esteem being “informed” by metrics

  • The intention is that under the REF all three areas will be metric determined.

  • This will require the introduction of bibliometrics into the REF.

The three most important words l.jpg
The Three Most Important Words

  • You might guess that the three most important words in the REF were:




Unfortunately, you would be wrong!

The three most important words4 l.jpg
The Three Most Important Words

  • Without any doubt the three most important words in the REF are:




The worrying thing here is that, at the moment, “WE” includes HEFCE!

Surely hefce aren t that stupid l.jpg
Surely HEFCE aren’t that stupid?

  • HEFCE have conducted a pilot study into gathering bibliometric data.

  • The pilot was to run concurrently with the 2008RAE to provide a basis for comparison.

  • Unfortunately this proved to be impossible – and so no benchmark is available!

The hefce bibliometric pilot study l.jpg
The HEFCE Bibliometric Pilot Study

  • Involved 16 universities

  • Covered 35 UoA’s – good coverage in; Science, Computing, Engineering. Very little social science or arts coverage.

  • The candidates were asked to provide bibliometrics – including citation data on all staff in the UoA which were submitted to RAE2008.

  • The candidates were asked to provide the information as quickly as possible, consistent with quality checking.

The hefce bibliometric pilot study7 l.jpg
The HEFCE Bibliometric Pilot Study

  • The “best” university managed to comply in 3 days!

  • One university took 150 days! – using an undisclosed, but substantial, number of staff.

  • In two years time we could be asked to do this on an annual basis!

Lessons learnt from the pilot l.jpg
Lessons Learnt from the Pilot.

  • Difficulties with older or existing information.

  • The process must be on-going and not just pre-submission.

  • Data cleansing is very time consuming – formats, mistakes, missing fields etc.

  • Difficulties linking publications to staff and staff to UoA’s (HESA id).

  • In some subjects the coverage simply is not there – i.e. the information does not exist!

Does this slow hefce down l.jpg
Does this slow HEFCE down?

  • Not a bit of it!

    • Autumn 2009 – consult on main features of REF

    • May 2010 – outcome of consultation, phasing in timetable, main operational features in place.

    • 2011-12 – metrics begin to inform funding.

    • 2012 – submissions to 2013 ‘light touch’ peer review process.

    • 2013 – undertake full assessment using ‘light touch’ peer review and metrics

    • 2014 – HEFCE research funding for all subjects driven by REF from this time on.

So what should we do l.jpg
So, what should we do?

  • JMU Working Party set up. This includes;

    • UoA co-ordinators from science/technical, social science and arts subject areas.

    • Colleagues from Computer Services and the Library.

    • Representatives from Research Office.

  • Decided that we must plan for the “worst case” scenario. That is, we will be required:

    • to return bibliometric data on ALL publications,

    • by ALL academic staff,

    • referenced back to UoA’s,

    • Over a 10 year period,

    • annually!

Introducing sympletic l.jpg

  • Sympletic is a publications management system developed in collaboration with Imperial College.

  • What Sympletic does:

    • We supply a list of names, aliases, email addresses, affiliations and UoA data.

    • Sympletic trawls databases looking for publications by these individuals.

    • If it finds a candidate publication it emails the person concerned and asks them to confirm if the paper is theirs.

Introducing sympletic12 l.jpg

  • On confirmation...

    • Sympletic will download all information regarding that publication.

    • Sympletic will maintain this information in an updated condition – particularly citation data.

    • Staff will be able to examine their own record and notify Simpletic of any errors or omissions.

Sympletic l.jpg

  • The plus points:

    • Once set up, the system is self maintaining.

    • Producing up to date reports at, multiple levels,

      • University

      • Faculty

      • School

      • Research Group

      • Individual

    • Can automatically update websites

    • Resource for REF, CV’s, Grant Applications, Corporate Publications etc.

    • Sympletic “talks” to Oracle and Oracle HR.

Sympletic14 l.jpg

  • The minus points:

    • We need a definite staff list – referenced to UoA’s

    • Staff engagement

      • Personal benefits

      • Managerial benefits

    • Issues with coverage – WoS and PubMed. Other databases?

    • Does not solve the problem of the data simply not existing.

Sympletic action points l.jpg
Sympletic – Action Points

  • Software Demonstration

  • If used the timetable would be:

    • Spring 2009 – Pilot

    • Sept 2009 – roll out

    • March 2010 – full deployment.

  • Management input needed to promote staff engagement by adopting Sympletic for other processes:

    • Profs/Readers

    • Website maintenance

    • Validation/Accreditation documents

    • PDPR