Potential Roles for Ex-Ante Economic Evaluation of Agricultural Research

1. Potential Roles for Ex-Ante Economic Evaluation of Agricultural Research Paul Heisey USDA-Economic Research Service WREN Spring Interactive Workshop June 6, 2008

3. There are well developed methodologies for the economic evaluation of agricultural research Alston, J.M., G.W. Norton, and P.G. Pardey. 1995. Science Under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting. Ithaca, NY: Cornell University Press • Same reasons for evaluation: • To guide program management • For accountability • Most commonly applied methods (econometric, economic surplus) have been retrospective • Analyses may be more credible at a more aggregated level

5. Formal methods of ex ante economic evaluation of research have been relatively rare in agriculture • Some economic surplus analyses have combined past costs and benefits with projected costs and benefits • Formal “scoring models” have been applied in the U.S., but have been criticized (Ruttan, 1982; Alston, Norton and Pardey, 1995) • The combination of economic surplus models with mathematical optimization techniques for ex ante evaluation has been rare—primarily applied to commodity research in the developing country context Janssen et al., 1990—CIAT Scobie and Jacobsen, 1992—Australian wool Nagy and Quddus, 1998—Pakistan Mutangadura and Norton, 1999—Zimbabwe Fuglie and Thiele, 2006—CIP

6. Most studies show high median rates of return to agricultural research • Literature reviews • Evenson, 2001 • Alston et al., 2000 • Recent summaries • Fuglie and Heisey, 2007: median estimated rate of return to public sector agricultural research, U.S., 45 percent • Gray and Malla, 2007: Canada, 30-50 percent • Fewer analyses of this type for other (non-agricultural) sectors, but results appear comparable • Cockburn and Henderson, 2000: rate of return to public funding of biomedical sciences in the U.S., 30 percent “may be an underestimate”

7. Agriculture has some distinctive characteristics that influence the generation and evaluation of research benefits • Biological nature of agricultural production • Reproductive characteristics of domesticated crop and livestock species • Importance of disease (shared with human health research) • Influence of weather • Firm size in agriculture • 2002 Census of Agriculture: mean sales of $94,245 per establishment • 2002 Economic Census, all establishments with payrolls: mean sales of $3.1 million • Atomistic characteristics of production agriculture generally do not carry over to input supply firms or output processing firms • Spatial diffusion of agriculture • As a result of these characteristics • Location specificity in research results • Individual production agriculture units unlikely to have incentives to do optimal amount of research • But, there is the question of when it is socially optimal for the public sector to do production agriculture research, and when it is socially optimal for research to be performed by private firms (input suppliers, output processors)

8. There are important areas of mutual interest and overlap between agricultural research and other research areas, e.g. environmental research or human health research • The non-market nature of many benefits in these areas makes measurement more difficult • Attribution problems, always an issue in research evaluation, become even more problematic • The biological, structural and spatial characteristics of agricultural production, along with the non-market nature of some agriculturally related research, suggest a “one size fits all” evaluation methodology may not be appropriate across all public sector agricultural research programs

9. The Agricultural Research Service (ARS) approach to research priority setting and evaluation has been strongly oriented to peer review Relevance ARS National Program Cycle Base Funding Reallocation to Priorities & Budget Development and Allocation of Increase Research Program Planning & Priority Setting Scientific Merit Peer Review of New Research Projects Quality (Retrospective) Quality (Prospective) Program Review & Assessment Project Implementation Program Coordination Source: Rexroad, 2008 Performance

10. Sources of Input to ARS Research Program Planning and Priority Setting Process* Executive Branch (OMB, OSTP, USDA, other Federal agencies) Congress ARS Program & Budgeting Priorities Customers, Partners, Stakeholders, & Advisory Boards Agency Scientists & Managers • Basic Unit of relevance planning (synthesis of inputs to determine priorities) is a National Program each of which is made up of multiple research projects that contribute in a coordinated and complementary way to achievement of 5-year objectives Scientific Community Source: Rexroad, 2008

11. Can informal Rapid Impact Assessment (RIA) methods bring economic reasoning to research priority setting? Developed by New South Wales, Australia, Department of Primary Industries • Collect information on: • Objectives of R&D • Alignment with priorities • Size of target industry sector • Naïve ‘with’ and ‘without’ scenarios • Nature of market failure • Naïve assessment of shares to beneficiaries • Budget information • Stakeholders share of funding • 3 Hours time limit (with research managers) • Incorporate in project approval process • Prioritization based on • Comparison of industry impact with industry research funding • Relative importance of research for human or environmental health • Importance of research for maintaining critical scientific skills in the public sector Source: Mullen, 2008