Writing Proposals, Getting Reviews, and Persevering

1. Writing Proposals, Getting Reviews, and Persevering Ming Tai-Seale, PhD, MPH School of Rural Public Health

2. Acknowledgement • Enola Proctor, PhD • Washington Univ at S.L. • Kenneth Wells, MD • UCLA

3. The Agenda • Life of a proposal • Scientific review: who, where, how • Critical areas for improvement • Summary statement • Finding your agent: working with program staff

5. Overall Strategies • Think First, Second, and Third • Get Feedback at all Stages (3 times) • Develop and Follow Timeline for Submit • Plan on 3 Major Rewrites • Talk to Agency Staff Early, 3 Times • After a Good Idea and Opportunity, It’s Methods, Methods, Methods -From Ken Wells 3

6. Finding Ideas • Make a list of 5 project ideas that you would want to work on for years • Prioritize and sketch out 1-3 ideas • Get feedback from senior researchers, clinicians, or patients and community members

7. Have a Conceptual Framework

8. Having Ideas • Make a list of 5 project ideas that you would want to work on for years • Prioritize and sketch out 1-3 ideas • Get feedback from senior researchers, clinicians, or patients and community members

9. Shaping the Idea • Based on literature, agency priorities, identify next steps suitable to your stage of development and institution capabilities • Identify data sources needed • Identify special opportunities • SENIOR CONSULT

10. Develop the Design • Clarify settings and subjects • Develop Partnership – What model of collaboration? • Consider diversity goals • CONSULT with a statistician • Hypotheses • Sampling • Design

11. Developing the Design (cont.) • Develop rough ballpark for budget • SENIOR HELP NEEDED • Discuss concept with funding agency, based on aim, opportunity, design, and likely budget

12. First Draft • Develop Conceptual Framework • Identify Main Variables • Draft Interventions, if applicable • Develop Pilot Data • GET SENIOR CONSULT AND BEG FOR $ • Write Aims, Background, Design (Pretend you’re almost done!)

13. First Draft • Statistician Consult to help outline main analysis and develop Power Calculations to determine: • Is the Study Affordable? • If Yes, PROCEED TO PARK PLACE • If Not, Go Back to Start (JAIL)

14. Second Draft • Develop Operations plans • team organization, data collection, timeline • Detailed Budget--GET HELP • Statistical Consult: Detailed Analysis Plan • Review assumptions • Don’t delegate blindly • They are YOUR hypotheses; • Modify design, scope, budget as needed • Human Subjects Section: Consult with IRB, Mentors • Plop revised draft together

15. Third Draft • Meet all agency requirements • Develop budget justification • Highlight “value added,” pilot data, fit of aims, method, and analysis to model • AGENCY & SENIORREVIEW

16. It’s Not Over Yet • Take the feedback seriously • Revise the whole proposal if needed • Common problems: Not feasible (budget too large or scope too broad ); No pilot data; Aims not specific; Background has literature but no synthesis, framework, value-added unclear; Methods are under-developed (alternatives not considered; analysis not tied to hypotheses; design flaws: wrong sample for aim, causal inference poor; limitations unaddressed

18. Disciplines Anthropology Biostatistics Economics Epidemiology Health services research Medicine Nursing Organizational Theory Sociology Methodological Orientations Quantitative Qualitative Mixed Stages in Their Own Careers Senior Scholars Emergent scientists Who Serve on Study Sections?

19. The Review • Source: Enola Proctor

20. Reviewer charge Assess likelihood that proposed research will have substantial impact on pursuit of NIH research goals: Advance understanding of • Biological systems • Improve control of disease • Enhance health

21. Significance Approach Innovation Investigator Environment Priority Populations Budget Human Subject Protections Summary Major strengths and weaknesses Recommendation for or against funding Review Criteria

22. Significance • Is the dependent variable important to NIH objectives? • Does the study extend boundaries of current knowledge? • Does the study address funding agency priorities? • Bridging Science and Services • Surgeon General Surgeon General’s report • IOM: quality • Agency partnership: “The Road Ahead” • Can the findings inform policy or practice?

23. Approach: Aims • Is the research question fully specified? • Aims are manageable, but reflects long term agenda • Incorporates relevant concepts, variables

24. Approach: Methods • Setting and context • Collaborative arrangements • feasible data collection plan (sufficient subjects, reasonable procedures, plan (sufficient subjects, reasonable procedures,good measures) • Manageable: • Sufficient time for methodology and products? • Person power: enough, and the right people • Analytic model = fully specified • rival explanations controlled through design or measurement of relevant variables • Procedures protect data quality • Appropriate methods of analysis • Sufficient power

25. Methods: Critical Areas for Improvement • Design problem • Measurement • Choice of variables • Intervention/comparison • Analysis problem • Choice of approach • Technique • Test • Theoretical or conceptual model or framework • Missing, deficient, or erroneous

26. Innovation • Original and innovative aims • Novel concepts, methods, approaches • Challenges existing thinking or approaches • Advances new methods or technologies

27. Investigator • Sufficient experience to direct the project • Well trained • Productive • Proposed project builds on background work • investigator publications • preliminary studies • Proposed work will make “next logical next logical” contribution to knowledge • Is surrounded by a team who can ensure success • Range of substantive and methodological expertise • Multidisciplinary • Critical areas for improvement • Investigator expertise deficient • Needs consultants or collaborators

28. Environment • Environment contributes to project success • Project capitalizes on environment, its people, and resources and resources • Collaborations • Evidence of organizational support

29. Ks Review Guideline • Candidate • Career development plan • Research plan • Mentor/co-mentor • Environment and institutional commitment • Budget • Human subjects • Women/minorities/children • SUMMARY • major strengths and weaknesses • Recommendation for or against funding

30. Overall Review • All criteria are considered when assigning overall score • Application does not have to be equally strong in all categories in all categories • Major issue: will proposed research have major scientific impact?

31. Rating Scale in NIH Review 1.0 virtually flawless, with negligible weaknesses 1.5extremely strong, with a few minor weakness 2.0very strong, but with moderate weaknesses 2.5 strong, but with some major weaknesses that must be addressed 3.0fair, neutral balance of strengths and weaknesses 3.5 weak, but with some major strengths

32. Priority Score • How is the summary priority score calculated? • Group average • Average Score times 100 • Equal weight • Some are “un-scored” • What is the fundable range? • Study sections can have different norms • When in doubt, ask the project officer

33. The Physical Setting

34. Critical Areas for Improvement in Rs • Design problem • Measurement • Choice of variables • Intervention/comparison • Analysis problem • Choice of approach • Technique • Test

35. Critical Areas for Improvement in Rs • Weak justification for study • Background and significance unconvincing • Literature review incomplete • Investigator expertise deficient • Needs consultants or collaborators • Theoretical or conceptual model or framework • Missing, deficient, or erroneous

36. Summary Statement

37. How to Read the Pink Sheet • Expect the language to be • Frank, and • Not overly enthusiastic • Be emotionally detached, after the initial… • Talk to an experienced grant-maker • Resubmit unless you see “fatally flawed” • Do NOT resubmit right away • Recruit a “cold reviewer”

38. Take a Vacation …

39. Program Staff • Project officer – Your Agent • Read your concept paper and draft • Send it in EARLY!, and • Often • Interpret the fundability of your priority score

40. Resources • Instructions on how to prepare your application • http://grants.nih.gov/grants/funding/phs398/section_1.html • Follow it VERY closely • Panel presentation at AcademyHealth 2006 • http://www.academyhealth.org/2006/demystifying2006.pdf