Analysis of indicators used for crc monitoring and evaluation ljubljana 15 september 2009
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

Analysis of indicators used for CRC Monitoring and Evaluation Ljubljana, 15 September 2009 PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Analysis of indicators used for CRC Monitoring and Evaluation Ljubljana, 15 September 2009. Toolbox CRC programme managers – Dag Kavlie, RCN. Analysis of indicators used for CRC Monitoring and Evaluation Ljubljana, 15 September 2009. Toolbox CRC programme managers – Dag Kavlie, RCN.

Download Presentation

Analysis of indicators used for CRC Monitoring and Evaluation Ljubljana, 15 September 2009

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Analysis of indicators used for crc monitoring and evaluation ljubljana 15 september 2009

Analysis of indicators used for CRC Monitoring and EvaluationLjubljana, 15 September 2009

Toolbox CRC programme managers – Dag Kavlie, RCN


Analysis of indicators used for crc monitoring and evaluation ljubljana 15 september 20091

Analysis of indicators used for CRC Monitoring and EvaluationLjubljana, 15 September 2009

Toolbox CRC programme managers – Dag Kavlie, RCN


Crcs have in some important aspects in common

CRCs have in some important aspects in common

Definition by COMPERA:

Structured, long termRTDI collaborations in

strategic important areas between academia,

industry and the public sector

Aim:

Bridge the gap between scientific and economicinnovation by providing a collective environment for academics, industry and other innovation actors and creating sufficient critical mass

Multiple activities:

Pooling of knowledge, creation of new knowledge by performing different types of research, training and dissemination of knowledge, networking, …


Crc programmes are different in some aspects

CRC Programmes are different in some aspects

Primary characteristics

  • Focus on Research as a knowledge basis for Innovationvs Innovation

    Other important features

  • Physical centre vs Network, (virtual centre)

  • Separate Legal Entity vs Centre within Host institution

  • Regional (national) focus vs International

  • Active participation in centre activities by enterprises vs cash contribution

  • Duration of funding, life after end of funding of centre

  • Size of Budget and Funding profile

  • Open call vs Predefined thematic area

  • Industry led vs Academic lead


Analysis of indicators used for crc monitoring and evaluation ljubljana 15 september 2009

Positioning of CRCs is crucial to understand the type of impact and time perspective we should expect

Yes

Goal

Scientific

progress

No

No

Yes

Goal

Economic or social use

Research in focus

Areas of New technology

Needs-driven

Basic research

(Pasteur)

Curiosity driven

Basic research

(Bohr)

Innovation

in focus

Industrial research

(Edison)

Increased ”end market” driven


Physical centres vs networks

Physical Centres vs Networks

  • For a Physical centre the researchers are located in the same premises

  • Virtual centres may also have research groups that are located together but at different locations

  • For Networks the researchers are generally located in the institutions that are the partners of the network


Some main features of present crc programmes

Some main featuresofpresentCRC Programmes


What is the purpose of indicators nsf

What is the purpose of indicators (NSF)

  • Collection of some standard measures of performance across all CRCs in a programme (Top down indicators)

  • Information base for Programme management

  • Information base to help centre Evaluators (May also include bottom up indicators for each centre)

  • Information base for Agency report to funding Ministries on Programme achievements


Crc australia measurements along the input to impact chain

CRC-AustraliaMeasurements along the input to impact chain

INPUT

  • People, Money, Infrastructure, Prior IP

    ACTIVITY

  • Research projects, Stakeholder engagement, Training

    OUTPUT (Result)

  • Publications, Prototypes, Patents, Ph.ds, Masters,

    IMPACT (Outcome)

  • Gains in Productivity, Industrial development, Health and Environmental benefits


Indicators should have a close relation to the goals for a centre

Indicators should have a close relation to the goals for a centre

Example of expected impacts (outcomes) :

  • Research in the forefront within thematic area

  • Knowledge basis relevant for industrial partners

  • Training of researchers in areas important for industry

  • Internationalisation

  • Increased R&D spending of business partners

  • Innovations by partners

  • Impact on industry and society at large


Example output outcome indicators for canadian nces

Example: Output/Outcome indicators for Canadian NCEs

Performance area 1: Researcher training and Recruitment

  • Number of Post docs

  • Number of Ph. D students

  • Number of Master students

  • Number of candidates from centre that are employed in industrial sector

    Performance area 2: Transfer and exploitation of results by industry

  • Number of patent applicatons and patents issued

  • Number of license agreements and income by licenses

  • Number of new products, services and processes

  • Case studies demonstrating impact

    Performance area 3: Increased productivity and economic growth

  • Number of jobs created

  • Examples of companies created in new industrial sectors

  • Case studies of impact on existing industries

  • Benefit/cost analysis


Analysis of indicators used for crc monitoring and evaluation ljubljana 15 september 2009

Management

and organisation

Innovation and Value creation

Research

Recruitment

International

cooperation

” Example of a performance areascovered in the midway evaluation for a Norwegian CRC Must identify relevant indicators for each of these areas


Some important observations

Some important observations

  • Useful with some top down indicators common for all centres and connected to the strategic aims of the programme

  • Allow centres to formulate their own bottom up indicators which are considered particularly relevant for each centre

  • The time domain must be taken into account

  • Indicators are only one element to be considered in an evaluation of a centre or programme

  • Case study of success connected to performance areas valuable part of an evaluation

  • A common European ”best practice” is not to be aimed for, even if competence centres have much in common

  • Much can be learned by study of the indicators used for different programmes together with the strategic aims of the programme


Analysis of indicators used by compera members

Analysis of indicators used by COMPERA members

Feedback from eight COMPERA members

Analysis along three main dimensions

  • Research

  • Innovation

  • Centre dimension


Research dimension result output indicators commonly used

Research dimensionResult (Output) indicators commonly used

  • No of approved EU-projects within the centre’s field of operation (6)

  • No of published papers in refereed journals (5)

  • No of international conference contributions (5)

  • No of projects with international partners (4)

  • No of Co-publications with industrial partners (3)

  • No EU-projects with role as coordinator (3)

  • No of M Sc degrees connected to the centre (3)

  • No of Ph.d students working in the centre (3)

  • No of international visiting researchers (2)


Innovation dimension result output indicators commonly used

Innovation dimensionResult (Output) indicators commonly used

  • No of Patent applications (5)

  • No of new enterprise partners (5)

  • No of project results that are protected by other than patents (trademarks etc)(4)

  • No of projects with active involvement of enterprise partners(4)


Centre dimension result output indicators commonly used

Centre dimensionResult (Output) indicators commonly used

  • Active involvement of enterprises in Research agenda setting (6)

  • No of centre events like workshops, seminars etc (5)

  • Volume of additional funding (4)

  • Communication - Press cuttings related to centre (3)

  • Mobility of Staff between partners (2)


Impact outcome indicators commonly used

Impact (Outcome) indicators commonly used

Research

  • No of Ph.d theses completed (3)

  • Increase in R%D spending by enterprise partners (4)

    Innovation

  • No of Patents (5)

  • No of licences based on patents (4)

  • No of new Products, Processes and Services (5)

  • No of Spin-of companies (4)

  • Recruitment of personel from academia to industry (4)


  • Login