AEA/CES Think Tank Operating in Pasteur’s Quadrant: Use Inspired Basic Research

1. AEA/CES Think TankOperating in Pasteur’s Quadrant: Use Inspired Basic Research Bhavya Lal Core Staff Member Science and Technology Policy Institute 1899 Pennsylvania Avenue NW, Washington DC 20006 Lisa Messeri Analyst, Abt Associates Inc 55 Wheeler Street, Cambridge MA 02138

2. Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

3. Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

4. The “Linear” Model of R&D Leading to Innovation Funding Societal Benefit Basic Research Applied Research Development Technology/Application Condensed Matter Physics Electronics Industry

5. Funding Societal Benefit Basic Research Applied Research Development Technology/Application Complex “Ecosystem” in which Innovation Flourishes Capital Markets Govt. Regulation Industry

6. Theoretical Constructs • Kline’s Chain-Linked Model (1985) • The central process of innovation is not science but design, and the assumption that basic research drives innovation is flawed (Kline & Rosenberg, 1986) • Leydesdorff “triple helix” (2000) • It is incorrectly assumed that universities and industry are the only players. Several other actors play important roles in innovation processes, such as government, investors, and end users • Stokes’s Pasteur Quadrant (1997) • There is an artificial distinction between basic and applied research http://nanoandsociety.com/ourlibrary/documents/bsts-nano.pdf

7. Research Inspired by Consideration of Use Stokes’s Model No Yes Use Inspired Basic Research (Pasteur) Pure Basic Research (Bohr) Yes Research Inspired by Quest for Understanding http://www.cspo.org/products/conferences/bush/Stokes.pdf Pure Applied Research (Edison) No

8. Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

9. Defining Use Inspired Basic Research • Basic research is defined as systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind. NSF Definition of Basic Research • Use-Inspired basic research is simply that – basic research motivated by ultimate application!

10. Fundamental Insights System Requirements Knowledge Base Fundamental Research Develop useful insights fromfundamental knowledge Defining Use Inspired Basic Research Environment Government/Regulators/Industry • Basic research is defined as systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind. NSF Definition of Basic Research • Use-Inspired basic research is simply that – basic research motivated by ultimate application!

11. Examples • United States • Most of National Institutes of Health • Parts of NSF – Engineering, Computer and Information Science and Engineering • Canada – National Centers of Excellence • Japan – JST/ERATO • Sweden – Competence Centres • Australia – Cooperative Research Centres • New Zealand – New Economy Research Fund

12. Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Why is it difficult to evaluate • Case Study • Discussion: Potential Solutions

13. Fundamental Question Is UIBR meaningful concept worthy of evaluating differently than other kinds of programs?

14. Evaluation Challenges - 1 • Programs tend to be unclear as to whether they are UIBR, and what that means • Is it enough for the research to be “inspired” by practical use? Or do users need to be engaged? • Are most programs today partially use-inspired? • Simply because that is how you can justify taxpayer-funded programs! Can one identify a UIBR program as one?

15. Evaluation Challenges - 2 • More stakeholders, all of whom have conflicting priorities • PIs want to focus on “publishable” research; may often be unaware of, ignore, or purport to meet program goals • Industry has more influence, and wants “widgets” • Government wants to: justify programs to taxpayers, and show links between funding and economic development How should the evaluation be designed differently as a result of this?

16. Evaluation Challenges - 3 • Complex pathways to success • Great variability across projects; different definitions of “success” • Process issues becomes as important to examine and model as outputs and outcomes How do you operationalize these concepts into meaningful indicators?

17. UIBR Follows Complex Pathways in the Innovation Spectrum

18. Environment Requirements Products and Outcomes Identify societal/marketneeds; define system and system requirements Technology Integration Systems Research Technology Elements System Requirements Integratingfundamental knowledgeinto enablingtechnology Technology Base Enabling Tech. Research Fundamental Insights Develop usefulinsights fromfundamentalknowledge Knowledge Base Fundamental Research Source: NSF ERC Program

19. Measures for UIBR • Partnerships • Stakeholder engagement • Creation of test beds, prototypes, instrumentation • Creation of intellectual property • Dissemination • Development of human capital (training) • Paper productivity • Paper/journal quality • Awards and prizes Cheaper/newer/better products

20. Evaluation Challenges - 4 • Project data (especially process-related data) is distributed across multiple sectors, and therefore more likely to be incomplete or inaccurate • Tendency to exaggerate/de-emphasize impact • Some data is outside of the projects; often data on use is confidential All of the data collection challenges of basic research and applied research PLUS additional challenges

21. Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion: Potential Solutions

22. Challenges in Evaluating the New Zealand New Economy Research Fund (NERF) Program

23. What is NERF? • NERF was established by the Ministry of Research, Science and Technology to fund high risk, novel research which could lead to the emergence of new industries in New Zealand. Researchers self classified their activities as basic research and proof-of-concept.

24. Evaluation Methodology • We began “traditionally” • Used an adapted linear model (Branscomb et al) • During field work, it became apparent that the projects were taking more complex paths to satisfy the criteria of “establishing new industry” • NERF exemplified UIBR • Our methodology needed alteration

25. “Platform” Representation

26. Challenge 1: Program Unclear if UIBR • NERF’s Ministerial notice stated four broad goals: Fund investigator-initiated basic research; build critical mass; develop advanced human capital and skills; create a sustainable knowledge platform • Researchers were not focused on the Ministerial Notice • To deal with the ambiguity, our evaluation created a working definition for each goal

27. Challenge 2: Stakeholders with Conflicting Priorities • Identified stakeholders • Researchers • Research organizations • Government officials • Venture capitalists • Industry leaders • Due to an insufficient program model, were unable to link back stakeholders needs to project performance

28. Challenge 3: Complex Pathways to Success • NERF’s Ministerial notice was written in broad terms • Researchers freely interpreted the notice, leading to a wide range in project goals • Were unable to assign a single or a small set of ‘success metrics’ to projects • Some projects reported large numbers of publications; others focused on IP creation • Did highlight many different styles of success in case studies and qualitative examples

29. Challenge 4: Project data was distributed across multiple sectors • Did not encounter this anticipated challenge • Did not have systematic data from all sectors • Collected qualitative data during site visit

30. Overview of the Think Tank • Evolution of the concept: Use Inspired Basic Research (UIBR) • Definition • Evaluation Challenges • Case Study • Discussion

31. Discussion Questions • Is the concept of UIBR relevant today? Are other paradigms (e.g. triple helix, innovation systems) superior? • Is pretty much every program these days UIBR? • Are there particular management issues with UIBR that are worth discussing? • Are the challenges that we encountered while evaluating UIBR unique? What challenges did we miss? • How would you overcome these challenges?

32. Concluding Thought “Some bargains are Faustian, and some horses are Trojan. Dance carefully with the porcupine, and know in advance the price of intimacy” http://nanoandsociety.com/ourlibrary/documents/bsts-nano.pdf