Science and Technology Policy and Research Activity Evaluation.

1. Science and Technology Policy and Research Activity Evaluation. Prof. Chris Phillips, Experimental Solid State Group, Physics Dept.,Imperial College London, UK.

2. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

3. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

4. The 'Seven Principles of Public Life’ Selflessness Holders of public office should act solely in terms of the public interest. They should not do so in order to gain financial or other benefits for themselves, their family or their friends. Integrity Holders of public office should not place themselves under any financial or other obligation to outside individuals or organisations that might seek to influence them in the performance of their official duties. Objectivity In carrying out public business, including making public appointments, awarding contracts, or recommending individuals for rewards and benefits, holders of public office should make choices on merit. AccountabilityHolders of public office are accountable for their decisions and actions to the public and must submit themselves to whatever scrutiny is appropriate to their office. Openness /Transparency Holders of public office should be as open as possible about all the decisions and actions that they take. They should give reasons for their decisions and restrict information only when the wider public interest clearly demands. Honesty Holders of public office have a duty to declare any private interests relating to their public duties and to take steps to resolve any conflicts arising in a way that protects the public interest. Leadership Holders of public office should promote and support these principles by leadership and example.

5. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

6. Managed vs. Responsive Mode ’ “Responsive mode” Agenda set by the community themselves, democratically Responds to new trends quickly. Allows meaningful collaborations to grow organically Difficult to get “Critical Mass” for technology intensive programmes Can end up with an incoherent research portfolio in smaller communities. Managed Imposes coherence on the portfolio Can make scientists move with the times Who chooses themes? (International opinion sought?) Can become very political, and result in outdated portfolio and poor value for money.

7. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

8. Anonymity vs. Transparency Any anonymity decreases transparency Negative effects can be partly reduced by careful record keeping (aliases/ code numbers etc.) so that at least abuses are traceable. “Fully Open” Applicants and referees known to each other and everyone else. Corrodes professional relationships Inhibits expression of honest opinion Very rare (US?) “Single Blind” Applicants are known but referees known only to the administration Can be open to misuse Practical to administer Probably the most common(UK) “Double blind” Applicants and referees both anonymous Difficult to implement in small communities Used in EU, possibly to negate political forces

9. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

10. Categories vs. Interdisciplinarity We can’t live without categories! Advances often happen when ideas get transferred from one sphere to another Administration needs a carefully thought out way of handling bids that span disciplines KshitiJ 2010, Kolkata

11. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks. NCSTE Kazakhstan April 2012

12. Fundamental vs. applied Some thoughts..... Perhaps 3% (?) of “fundamental” research will lead to immense benefits for society. In the past, no-one has been able to forsee which 3% this will be (e.g. Lasers, Transistors, Nuclear Magnetic Resonance). Translation times are long ( 20 yr). To get the 3% you have to pay for the other 97%. If you fund only research that will be applied in two years, then in two years you’ll have no new ideas coming through. Politicians/Industrialists like applied research (because they’re not here for long). Political changes (e.g. Towards greater democracy) can put pressure on fundamental research funding. NCSTE Kazakhstan April 2012

13. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

14. Risk Vs. Reward • They are inseparable! Reward. Risk. Good projects ALWAYS carry risk.

15. Risk Vs. Reward • They are inseparable! • Rewards come in many forms. (Commercial, intellectual, cultural). • Risk SHOULD be there, and referees need to be educated carefully. • Risk should be managed (contingency plans, review points, intermediate deliverables). • Unless the funding system accepts and embraces risk, its research portfolio will never reach international standards. Good projects ALWAYS carry risk.

16. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

17. Costing models • “Full Economic Costing” (FEC) aims to capture the total cost of a programme including e.g. Coast of providing and maintaining buildings, covered in “Research Overheads” • Many bodies (e.g. UK charities, EU programmes) don’t pay full FEC rates, and boast of budgets they’ve “leveraged” from other sources. • FEC costing certainly helps referees judge a proposal properly. Especially overseas referees. • “Leveraged” programmes are often under resourced sometime on a small but critical area (“spoiling the ship for a haporth of tar”). • Resources need to be justified and appropriate.

18. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.

19. Impact and Public Engagement • Most science, and much technology is funded publicly. • We probably, at the least, have a moral duty to give as much back to them as possible. • In a democracy, funding agencies have the much more practical imperative to document their impact, or the money is likely to dry up. • “Impact” comes in many forms........... • Commercial Impact? • Trained staff/students. • Scientific Impact/glory? • Cultural value/advances in knowledge? • Ca the discoveries impact on different fields/disciplines? • Will the discoveries lead to progress in areas identified by the government/funding body? • How will the general population (and politicians!) hear about the work? (media, TV, books etc).

20. Outline Introduction/Overview Managed vs. Responsive Mode Anonymity vs. Transparency Categories vs. Interdisciplinarity Fundamental vs. applied Handling Risk Costing models Impact and Public Engagement Concluding Remarks.