The Grant Writing and Review Process at NIH

1. The Grant Writing and Review Process at NIH Joshua Smyth Professor of Biobehavioral Health and Medicine; Associate Director, SSRI Rhonda BeLue Associate Professor of Health Policy and AdministrationJenaeNeiderhiser Professor of Psychology Danielle Downs Associate Professor of Kinesiology and Obstetrics & Gynecology

2. Thanks to: • Lori Francis, Associate Professor of Biobehavioral Health • Brittany Frost, Social Science Research Institute

3. Workshop Outline • NIH Organization • NIH Funding Mechanisms • The Grant Writing Process • Focus on the R01 • The NIH Review Process • Overview of Review Meeting • The Scoring Process • A Penn State example • Workshop Evaluation

4. I. The NIH • Department of Health and Human Services • National Institutes of Health • 25 Awarding Institutes/Centers aka ICs • e.g., NICHD, NIMH, NIDA, NIA, etc. • Center for Scientific Review • Office of the Director

5. The NIH Extramural Team A. Program C. Review B. Grants Management

6. A. The Program/Institute StaffProgram Administrator • Maintains knowledge of scientific area • Attends study section meetings • Makes funding recommendations • Monitors scientific progress • Identifies scientific area of importance • Reports to senior staff • Development of programs and initiatives

7. B. GrantsManagement • Interprets Federal regulations and policies • Assures compliance with Federal regulations and policies • Monitors financial aspects of projects • Interprets regulations and policy

8. Center for Scientific Review (CSR) or NIH Institute & Center (IC) Scientific Review Group (SRG) Non-federal scientists with relevant expertise Led by a Scientific Review Officer (SRO) http://www.csr.nih.gov/Roster_proto/sectionI.asp C. Review:Scientific Review Group (1st Level)

9. C. Review:Advisory Council or Board (2nd Level) The potential awarding IC performs the second level of review NIH program staff examine applications for impact (formerly “priority”) scores, percentile rankings, & summary statements against the IC’s needs Program staff provide grant funding plan to Advisory Council or Board Advisory Council or Board advises the IC director Director makes final decision

10. NIH Grant Application Cycle Investigator Institution NIH InitiatesResearch Peer Review Submits Application Council Review Allocates Funds Funding Decision ConductsResearch

11. Grant Application: It’s a process, not an event • Communicate with Program Officer • Introducing ideas, getting feedback, pre-review 2. Get your proposal to the right review committee • Review the rosters and talk to colleagues • Effectively wording the abstract • Make a written request 3. Seek feedback from colleagues and consultants on drafts of the grant (prepare ahead!) 4. Consider who is likely to review your grant (review the rosters) and make sure to know and cite their work when relevant 5. Recognize that funding on first submission is rare

12. II. NIH Grant Mechanisms • Ks: NIH Career Development Awards (K01, K02, K05, K07, K08, K22 [K99/R00]) • P01: Research Program Project Grant • P30: Center Core Grants • R01: NIH Research Project Grant Program • R03: NIH Small Grant Program • R13: NIH Support for Conferences and Scientific Meetings (R13, U13) • R15: NIH Academic Research Enhancement Award (AREA)

13. NIH Grant Mechanisms (continued) • R21: NIH Exploratory/Developmental Research Grant Award • R34: NIH Clinical Trial Planning Grant • T series: NRSA Training Grants (T32, T34, T35, T90, etc.) • U series: Research Project Cooperative Agreement • Diversity Supplements: Research Supplements to Promote Diversity in Health-related Research • Roadmap: NIH Roadmap Initiatives (Director’s Pioneer Award; Director’s New Innovative Program)

14. R01 (primary focus today) • Used to support a discrete, specified, circumscribed research project • NIH’s most commonly used grant program • Body of the grant (Research Strategy) is 12 pages (mostly single spaced) • Generally awarded for 3 to 5 years • Up to $500,000/year without exceptions needed • Advance permission required for $500,000 or more (direct costs) in any year

15. New and Early Stage Investigators:A Competitive Edge • New Investigator has not previously served as a PI for an R01; may have been an investigator or received other smaller, developmental or research training awards • Early Stage Investigator (ESI) is within 10 years of completing her/his terminal research degree, or completing medical residency

16. III. The Grant Writing Process • Grant writing is: • A skill like any other… • But not the same skill as article writing • Instead, more of a problem-based writing activity (theory and practice problem)

17. A few preliminary tips • Start early, make a timeline and STICK TO IT • Should allow time for serious pre-submission review and subsequent revision • Develop a relationship with project officers • It is not possible to overdo clarity • Let your passion come through in your proposal • Take advantage of early stage and new investigator opportunities

18. Getting ready to write • Know what has been done • Know what has been funded • NIH website RePORTER (formerly CRISP) • Decide on the problem • Important enough to get funded but simple enough to explain as clean design • in 12 pp for the R01; less for some other mechanisms • Assemble team • Complementary skills; Seniority/competence/other by association; People you can count on

19. Getting ready to write • Communicate with program officer • Establish a relationship and trust (funding decision) • Acquire information on mechanism and priorities • Obtain input on aims/proposal

20. Main Sections of the NIH Application(see Francis et al. for some examples) • Face Page • Table of Contents • Performance Sites • Other information • Project Summary (Description) • Public Health Relevance Statement • Facilities & Resources

21. More Sections • Key Personnel • Biosketches-- with personal statements • Budgets (for each study year) • Budget Justification • Other sections (not discussed today); for example: • Clinical Trial and Human Embryonic Stem Cell (HESC) • List of Research Plan Attachments

22. Main Sections of the R01: Specific Research Plan • [Introduction – revisions only] • Specific Aims: The basis for the proposal’s organization • Research Strategy • Significance and Innovation • Approach • Preliminary studies • Design • Sample/recruitment/power analyses • Procedures & measures • Analyses

23. Page Limit Guide: Plan your proposal with these limits in mind Page limits may vary for other funding mechanisms. Check Funding Opportunity Announcement: http://grants.nih.gov/grants/guide/search_results.htm?scope=pa&year=active

24. And More Sections • Protection of Human Subjects • Women and Minorities • Planned Enrollment Table • Children • References Cited • Letters of Support • Resource Sharing Plan • Checklist SO START EARLY!

25. General themes of success • Clarity is key – each point follows naturally • Tell a (scientific) story • Everything that needs to be there, is; nothing extra • Communicate your excitement • Get feedback early on • And often

26. General themes to R01 Success • Impact and Significance • Practice (2-3 sentences) • Prevalence of problem in population • Important social concern/public health problem • Theory (model) • Building, testing, using • Innovation • New directions, value added • Compelling Preliminary Research

27. Keys to Success – Specific Aims • Specific Aims may be only part read by some reviewers • No more than one page • Tell the entire story • Functions as an abstract would in a manuscript • End with (actual) specific aims • Hypotheses and aims must align

28. Keys to Success – Background • Background and significance • Used to be an explicit section; no longer • This is where you ‘hook’ the reader on your story • Functions as the ‘introduction’ does in a manuscript • Likely no more than 2-3 pages

29. Preliminary Studies • Research team collaboration no longer required to be documented • Still very helpful • Preliminary study ‘types’ – document: • Relevant previous research • Research that demonstrates competence in requisite domains • Capacity to recruit in specific populations and/or contexts • Pilot data specific to proposal

30. Keys to Success - Methods • Methods are very important • Overall -- clarity and detail • May include a table that traces aims to hypotheses to constructs to measures (table/s) • Is the design feasible? • Are there gaps in the methods (e.g., fidelity for interventions) • Statistics are essential – product must match aims • Consider a methodologist team member • Include a detailed timeline

31. And More Methods: The Sample • Preference for representative samples • Students only if relevant to age/situation (e.g., college drinking) • Generalizability from a single entity (university, clinic, state) • Justified exclusions • Unit of assignment is unit of analysis • Sample size/power analysis • For each outcome/planned test

32. Budgets • Direct Costs • Senior Personnel (PI, co-Is, project director) (PSU fringe at 36.5%) • Other Personnel (full time staff, RAs, part time wages) (PSU fringe 36.5% for full time staff, 13.2% for Grad Assts AY; 7.9% for part time wages and summer) • Equipment • Travel • Participant/Trainee Support Costs

33. Budgets, cont. • Other • Materials and supplies • Publication costs • Consultant services • Subawards/Consortium/Contractual Fees • Other • Indirect Costs (~50% at PSU, but does not include all expenses) • Budget justification

34. Receiving the Summary Statements: The Hardest Part! • Reviews critical, even harsh • Reviewers usually find grant’s weaknesses, while recognizing strengths • Summary statements spend much more time on critique than praise • Many investigators experience a mixture of rage and depression when they read their summary statements and easily lose perspective • Take a day or two (or more!) and then read again with a cooler head

35. Receiving the Summary Statements: Bouncing Back! • Ask experienced colleagues to read reviews • Don’t interpret criticism as hopeless • Program Officer may be helpful in clarifying critique • If “discussed” (rather than triaged), you have a chance of funding in next round • The lower the initial score, the fewer problems and more likely to be successful after revision

36. Resubmission:Resilience and Flexibility! • Persistence pays off in the grant process!! • Second submission must respond to the critiques through revision or clearly defending reasoning • Same reviewers may or may not review resubmission, but will see critique

37. Most Common Reasons for a Poor Score (in priority order) • Lack of impact or significance • Lack of new or original ideas • Hypotheses ill-defined, superficial, lacking, unfocused, or unsupported by preliminary data • Methods unsuitable, not feasible, not rigorous or not likely to yield results; methods don’t clearly link to aims • Design not logical, inappropriate instrumentation, poor timing or conditions; doesn’t link well to aims • Data management and analysis vague, not rigorous; analyses don’t clearly link to aims • Inadequate expertise or knowledge of field for PI; too little time to devote to the work • Poor resources or facilities; limited access to appropriate population

38. When to Revise • Basic idea was significant and innovative or these can be bolstered • Design/measurement/analysis problems can be clarified (more information) or fixed • Need preliminary data • Problem is poor writing

39. IV. The NIH Review Process • A. The Review Meeting • B. Review Discussion • C. The Scoring Process • D. A Penn State Example

40. A. The Review Meeting: The SRO’s Role Prior to Meeting • Point of contact until review group meets (then project officer) • Analyze submissions for completeness and conflicts • Recruit ad hoc reviewers as needed • Schedule 1-2 day meeting • Assign applications to reviewers (at least 3) • Primary, secondary, discussant • Create review order based on preliminary impact scores from best to worst within categories

41. Reviewers’ Role Prior to Meeting • Familiarize self with criteria, mechanisms, and scoring • Review assigned applications • Assign scores to each criteria and other areas • Write bulleted strengths and weaknesses for each criteria • Reviews are advice to institutes for funding decisions, not advice to PI • Post scores and comments on NIH Commons • Read other reviews of assigned applications • Prepare presentation of reviews • Skim/read non-assigned applications

42. Format of the Review Meeting • SRO opening remarks • Chair orientation • New investigator R01 grants • Other R01 grants • Other grant types (R03, R15, R21, R34) • Applications discussed in order of Impact Score; bottom 50% are not discussed

43. SRO Opening Remarks • Confidentiality • Review order • Proposals below median within each category may not be discussed

44. Chair Orientation • Start with reviewer impact scores • May differ from posted scores • Goal of discussion is to clarify not reach agreement • If scores are similar, shorter discussion • If scores are dissimilar, longer discussion • Recommended time • Primary – 5 minutes • Secondary – 3 minutes • Discussant – 2 minutes

45. B. Review Discussion • Identify proposal • Members in conflict leave • Reviewers provide preliminary impact scores • Reviews • Impact, Significance, Investigators, Innovation, Approach, Environment • Stress main points, do not repeat previous points • Non-reviewers typically ask questions to clarify • Human Subjects issues affecting scoring • Open discussion to entire committee

46. Review Discussion (continued) • Ask for reviewers impact scores again • Identify the reviewers’ recommended range • Ask if anyone wants to score outside the range • Entire committee records impact score • Discuss budget and other issues

47. 1. Overall Impact Score: likelihood project will “exert a sustained, powerful influence on the research field(s) involved (1-9 scale) 2. A separate 1-9 score for each of 5 core criteria (Significance, Investigators, Innovation, Approach, Environment) 3. Additional review criteria help determine scientific and technical merit BUT are not scored separately 4. Additional review considerations are addressed by reviewers, but are not scored & are discussed after group scores C. The Scoring Process

48. Overall Impact: will project exert a sustained, powerful influence on the research field(s) as indexed by 5 core review criteria 1. Significance: important problem addressed; how will this improve scientific knowledge, technical capability, and/or clinical practice Score Criteria

49. What is the Difference Between Impact and Significance ? • Impact Addresses: • Probability of whether the research will exert a sustained, powerful influence on the research field • Significance Addresses: • Does the project address an important problem or a critical barrier to progress in the field? • If the aims are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved?

50. 2. Investigators: PI & other researchers well suited to the project; appropriate experience & training; ongoing record of accomplishments; complementary & integrated experience; leadership approach, governance, and organizational structure appropriate for project 3. Innovation: the work challenges and seeks to shift current research or practice paradigms; utilizing noveltheory, approaches or methods, instrumentation, or interventions; the work is novel (Be innovative, but maybe not too innovative…) Score Criteria (continued)