How Not to Write an Unsuccessful Grant Proposal

1. How Not to Write an Unsuccessful Grant Proposal Daniel Clauw, M.D. Assistant Dean of Clinical and Translational Research Director, Center for the Advancement of Clinical Research The University of Michigan

2. What is the end result you are aiming for?Good science by a good investigator. - Science + 25% of applications: Good idea, but Investigator is unprepared to do the work. 50% of applications: Bad idea, and poorly prepared investigator. 15% of applications: Great idea, proposed by the perfect person to test it. - Investigator + 10% of applications: Bad idea, but capable Investigator. Table developed by James Ferrara, M.D.

3. How to Get a Grant • Design a sound study • Write a good grant

4. Early Steps in Clinical Research • The idea • The hypothesis • The design • The study subjects • Can I do it?

5. The idea • Most clinicians can come up with good ideas about clinical research based on gaps in existing knowledge • Has question been asked before? • If not, why? • If so, how does my question or design add to the existing literature?

6. At the Beginning . . . Think of the End • At the inception of a clinical research project, ask a series of questions about the end product: • Who is the audience? • Where would the article be published if it is successful? • Is a negative study interesting or publishable? • How would the results impact clinical care • i.e. Who would care?

7. The First in a Series of Compromises

8. Potential audience • Academics • Peer review process • Practicing clinicians • Regulatory agencies (e.g. FDA) • Industry (marketing bias) • Public

9. Early Steps in Clinical Research • The idea • The hypothesis • The design • The study subjects • Can I do it?

10. Developing the Study Hypothesis / Goal(s) • A surprising number of even established clinical researchers begin designing the study before they define the hypothesis • Keep It Simple (KISS) • Unless you are in the position to do a hypothesis-generating study, have a single primary hypothesis and no more than 2 – 3 secondary objectives

11. Early Steps in Clinical Research • The idea • The hypothesis • The design • The study subjects • Can I do it?

12. Compromises in Study Design Efficacy Effectiveness Long trial Short trial Single site Multicenter Randomized, Blinded Not Active comparator Placebo Open

13. Early Steps in Clinical Research • The idea • The hypothesis • The design • The study subjects • Can I do it?

14. Inclusion and Exclusion Criteria • Might be most important aspect of study design with respect to: • Ability to recruit subjects • For clinical trials, this will largely determine: • Efficacy of treatment • Placebo response rate • For mechanistic studies, this needs to be considered for both patient (clearly define) and control groups

15. Selecting Participants Heterogeneous Homogeneous Extrapolatability of resultsSmaller n, more responsive Tertiary Care Population Easy to find / unresponsive High placebo response

16. Selecting Controls • Should ideally have all of the characteristics of the patient, except the condition being studied • Not always possible • Controlling for confounds • Sampling • Statistics • What is a “healthy normal control”?

17. Recruitment • Advertising • Apples vs. oranges • > Screen failures • Higher placebo and drop-out rates • Existing patients • Easy to find • Pre-screened • Stay in study longer

18. Early Steps in Clinical Research • The idea • The hypothesis • The design • The study subjects • Can I do it?

19. ResourcesDo I have enough? • Mentorship • Individual • Institutional (K-30, K-12) • Patients / subjects • Do I have enough? • Do I have the right ones? • Staff • Money • Federal grants • Institutional resources (e.g. GCRC) • Foundations • Industry

20. How to Get a Grant • Design a sound study • Write a good grant

21. Get FREE Money from the Government!!! OK, so maybe you don’t need this guy’s book. But you need some of his skills . . . . .

22. Besides good research, you need… • Communication Skills • Marketing Skills • Management Skills • Flexibility and ingenuity (who can you sell your research to?) • Ability to follow directions, even the apparently meaningless ones • Ability to plan ahead and commit significant time Adapted from Christine Black, UM DRDA

23. When? Earlier than you think • Internal vs. external deadlines • Should get science of grant done in plenty of time to have others read final grant, and the best people to read grant have the least time to do so • Administrative shell of grant typically needs to be turned in 10 days before grant is due • Study design issues, and especially sample size calculations, are necessary for administrative shell • Need for preliminary data

24. Do your homework: • What has the agency funded recently? (CRISP, Community of Science, www.guidestar.org for foundations). • Read recent successful applications by colleagues. • The program officers are your resource, contact them early and often. • Colleagues are essential for collaboration and consultation; senior colleagues may know who will likely review your grant. • Consult a biostatistician early in the process.

25. Do your homework, part 2: • Check study section rosters! • Search the literature to determine roster members’ expertise. • If there are content area experts on the study section, does your literature review cite their (appropriate) publications?

26. Do your homework, part 3: • Are their aspects of your project or methodology that may need extra justification because they will be unfamiliar to this roster of people? • For foundation grants, trustees or boards may make funding decisions, so be sure that key sections like specific aims are in lay language. • P.S. Do not attempt to contact reviewers! Inouye, S.K., Fiellin, D.A. An Evidence-Based Guide to Writing Grant Proposals for Clinical Research, Ann Intern Med. 2005;142:274-282

27. Relationships are key: • Step 1: Get the right team together. Find people with expertise in different, specific areas: working with this specific patient population, using this specific methodology or technique, statistically analyzing this type of data, etc. • Step 2: Buy your grants administrator a nice gift. You will be depending on this person!

28. Why you should be nice to your grants administrator: “This is the easy part. The challenge will be figuring out the indirect cost rate.” Chronicle of Higher Education, Carol Cable

29. Care and feeding of the reviewer: • They are… • Overworked (they will at most spend a few hours reviewing your grant) • Mindful of the need to further their own careers • Not interested in doing outside ‘homework’ in a new topic area just so they can understand your proposal • Mature • Translation: they don’t want to strain their eyes deciphering figures and captions that you’ve shrunk down to 8-point font! Adapted from Christine Black, UM DRDA

30. Care and feeding of the reviewer, part 2: • Assume reviewers are intelligent and savvy about research… • ..but perhaps have little in-depth experience in your area of interest. • Avoid jargon and topic-specific abbreviations. • Many reviewers will read only the abstract and/or specific aims. Inouye, S.K., Fiellin, D.A. An Evidence-Based Guide to Writing Grant Proposals for Clinical Research, Ann Intern Med. 2005;142:274-282

31. Care and feeding of the reviewer, part 3: • Make reviewer like you and your study in first two pages so that you start with a 1.0 • Teach them something new • Convince them that you are the best person in the world to do this terrific study • Science Fiction Novel approach – the world will be a better place after this study is done • Disneyland approach – make them smile while you are sucking money out of their wallets • If you can start with a 1.0 and eliminate “targets” you’ll be fine

32. “Targets” • Controversial aspects of a grant • It’s OK to “do it, but don’t write it” • Omissions • Failure to follow “rules”

33. Common Problems in Applications (from NIH and NIAID) • Lack of knowledge of published relevant work • Lack of experience in the proposed methodology or techniques • Uncertainty concerning future directions • Unrealistically large amount of work • Failure to discuss potential obstacles or alternative approaches Adapted from Christine Black, UM DRDA

34. Major issues identified in NIH grant proposals: • Results from a review of ‘pink sheets’ from 66 R01 applications; the authors categorized major problems found in unfunded grants. • Findings presented for each major grant section. Inouye, S.K., Fiellin, D.A. An Evidence-Based Guide to Writing Grant Proposals for Clinical Research, Ann Intern Med. 2005;142:274-282

35. Abstract • The broad, long-term objective is . . . . • The Hypothesis is . . . . • The Specific Aims are . . . . • The Research Design is . . . . • The health-related relevance of this project is . . . . • Avoid • First person • Excessive summary of past accomplishments • Amounts of money • Going outside the box

36. Specific Aims/Hypotheses • This (and abstract) are most important parts of a grant. • Write this early in the process, send it around, and keep re-writing to make it clearer, stronger, and more concise. • This should be a two page executive summary of the following 23 pages of the grant. • Writing tips: • ½ to one page of “setting the stage” • Extremely clear, (one sentence if possible) hypothesis • Specific aims (between 2 and 4) stated as action items • To determine, evaluate, confirm, show . . . . • Common critiques: • Poorly Focused • Too Ambitious

37. Comments from NIH Reviewer “In addition to proposing a research design that is a fishing expedition, the applicant also proposes to use every type of bait and piece of tackle known to mankind.” Adapted from Christine Black, UM DRDA

38. Special advice for Training (e.g. K-) awards • “Safe” study is more important than ground-breaking science. • Should be a study that leads to other studies, no matter what the results. • Serves as a vehicle for your training. • Should ideally encompass all of the elements of a research project, especially those that you have inadequate experience with. • Environment • Increasing preference for mentoring “teams”. • Letters are extremely important, and read for any nuances. • If your mentor cannot take the time to write a strong > 2 page letter, they may not have the time or commitment to mentor you.

39. Background and Significance • Begin by describing current state of science. • Identify knowledge gaps. • Justify the need for this specific study. • Move science forward, not just laterally. • Writing tips • Don’t provide too much extraneous background information. • Don’t overstate the significance of your study. • For each area covered in this section, explicitly state the relationship to your proposed project.

40. Comments from an NIH Reviewer “This application is characterized by ideas that are both original and scientifically important. Unfortunately the ideas that are scientifically important are not original and the ideas that are original are not scientifically important.” Adapted from Christine Black, UM DRDA

41. Preliminary Studies • Should demonstrate: • Feasibility • You can recruit the required study participants. • The team has successfully worked together. • Promising data • You’ve partly done what you are asking for money to do. • If there are novel methodologies that are being used, that these have been fully developed • Writing tips: • Abstract of each study is good start. • Show how each study links to the proposed work.

42. Methods • At least 50% of the page allowance should go to methods. • Underdeveloped or vague methods are the most common reviewer complaint. • Describe the study design in detail – howwill you randomize, blind, select controls, etc. • Inclusion/exclusion criteria – justify your criteria and address any biases they may cause.

43. Methods, part 2 • Availability of participants – • esp. with regards to women, minorities, and children. • Data collection and procedures: • Discuss each instrument you propose to use • Describe quality assurance or staff training • Consider a table of variables:

44. Interventions and Outcomes • Ensure adequate randomization and blinding. What will happen if the blind is broken? • Ensure standardization of your intervention. Will you provide training? • Ensure your intervention is meaningful. • Ensure that adherence is monitored. • Clearly define all outcome measures.

45. Statistical Analysis • Work with your biostatistician often and early. • Address what you will do about missing data. • Include realistic attrition rates in your power calculations. Address what you will do with data from subjects who drop out. • Present the plan clearly, referencing each specific aim when appropriate.

46. Writing style • Business not creative • Active voice • “will” or “expect to” not “would like to” or “may” • Anticipate questions • “Target” analogy

47. Final Touches • Realistically assess weaknesses or potential roadblocks and explain how you would overcome them. Don’t blow off this section – it’s your opportunity to think strategically. • Provide a timetable or chart summarizing study activities. • Proofread. A lot. Then give it to others. • Follow all the directions with regards to fonts, margins, page limits, deadlines, etc. • Finish with enough time for administrative approval, signatures, etc. – check with your grants administrator. This process may take 10 days or more!

48. How to deal with rejection:(You’re playing baseball, not shooting free throws) • Write scathing letter of rebuttal. • Throw it away. • Start with a paragraph of gratitude for reviewer’s insight. • List reviewer’s criticisms. • Do not try to convince reviewers that they are idiots • Address each criticism objectively in the introduction to your resubmission. Adapted from Christine Black, UM DRDA