Panelists

1. The R01 Grant Application Writing with Impact January 10, 2013 ARC, Room123-ABC

2. Panelists Janis K. Burkhardt, PhD.Associate Professor, Department of Pathology and Laboratory Medicine Douglas A. Coulter, PhD.Professor, Department of Pediatrics, Division of Neurology Andrea Kelly, MD, MSCE. Assistant Professor, Division of Endocrinology and Diabetes Sheela N. Magge, MD, MSCE. Assistant Professor, Division of Endocrinology and Diabetes Victoria A. Miller, PhD.Assistant Professor, Department of Anesthesiology and Critical Care Medicine Berenice Saxon, Sponsored Projects Office, CHOP Research Institute Virginia A. Stallings, MD. Professor, Department of Pediatrics, Division of Gastroenterology, Hepatology and Nutrition

3. Overview • Review Process and Scoring System • Writing the Grant • Advice for Clinical Researchers • Submitting the Grant • Interpreting Summary Statements • Deciding Whether to Re-submit • Responding to Reviewers • Lessons Learned as New Investigator • New Investigators and Early Stage Investigators

4. Scientific Review Group (SRG) “Study Section” • Chair • Moderates discussion • Serves as reviewer • Scientific Review Officer (SRO) • Assigns applications to reviewers • Attends and oversees administrative and regulatory aspects of peer review meetings • Prepares Summary Statements for all applications

5. Scientific Review Group (SRG) “Study Section” • Members (n ~ 20 to 30) • Receive grant applications six weeks before meeting • Prepare a written critique • Assign a numerical score to each of five review criteria (next slide) • Make recommendations concerning the scientific merit of applications Note: R01s can be triaged and not discussed at all Resource: Video of Mock Scientific Review Group

6. Review Criteria • Significance • Investigator(s) • Innovation • Approach • Environment (detailed in handout)

7. Impact v. Significance Overall Impact: the synthesis/integration of the five core review criteria of which Significance is one. “Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the five core review criteria, and additional review criteria (as applicable for the project proposed).”

8. Impact/Priority Score • 9-point rating scale (1 = exceptional; 9 = poor) • Score: mean of all members' impact/priority scores X 10. 10 (high impact) to 90 (low impact) • Numerical impact/priority scores are not reported for applications that are not discussed. Such applications typically rank in the bottom half.

9. Writing the Grant • Lead the reviewer through your proposal. • Organize the proposal in logical order. • Keep it simple, but address the concerns. • Be sure to address all the NIH review criteria. • Don’t neglect the “extra” sections.

10. Lead the reviewer through your proposal Organize your argument logically • Make sure your big ideas come across. • Do: Capitalize or italicize the central hypotheses. • Don’t: Lose the forest in the technical trees. • Think carefully about order of aims. • Does Aim 2 hinge on the success of Aim 1? • What is the most efficient and interesting way to lay out your questions? • Sometimes these are in opposition! • Where possible, show interrelationships. • Don’t be afraid to refer back and ahead. • Try to balance complexity of aims. • Don’t give them one to cut. • Sometimes they just aren’t balanced – address this. • Make clear what you expect to learn, and how it will advance the field. • Sometimes this means describing long term plans.

11. Lead the reviewer through your proposal Organize your argument logically • Picture the tired reviewer, with a stack of these things, who knows that only 1-2 favorites will likely be funded. • Keep writing clear and concise. • Take care to lay out your argument logically, so that the “bones” are readily apparent. • Take care with page layout. • Choose fonts of a reasonable size. • Leave spaces. • Intersperse figures.

12. Page Layout Dos • Use a systematic outline with consistent headings and subheadings. • Place figures near text they support, and use to break up the text. • Make the document do what you need to do. Create a consistent style with repeating special sections. • Consider dispensing with experimental systems up front. • Consider an introductory paragraph at the beginning of each aim or sub-aim to tell the reviewer what to expect later on. • Consider ending each section with a paragraph like “Pitfalls and alternative approaches”.

13. Page Layout Don’ts • Don’t jam the page full of text. • Reviewers are mean when they’re tired! • Don’t capitalize half the grant. • Reviewers don’t like to be shouted at! • Don’t try to put science where it doesn’t belong (e.g. in Vertebrate animals). • There are people whose whole job it is to send grants like that back! • Don’t try to get clever with size 10.5 font, etc. • They really do check! • Exception – figure legends.

14. Use Figures Wisely • Keep them simple. • Make sure there is a figure legend and they are referred to in the text. • Don’t assume that if it’s published, they will see it. • If your proposal hinges on a piece of data – show it. • Don’t overlook the power of a diagram, table or flow chart. • It can be a big space saver, and can really put your big ideas into perspective.

15. (C) APPROACH Interaction of T cells with APCs induces dramatic remodeling of F-actin at the site of TCR ligation, in a process that is required for productive T cell activation. Understanding how the actin cytoskeleton facilitates T cell activation requires identifying the relevant regulatory molecules, and determining how they work together to generate the appropriate actin architecture at the appropriate place and time. Our overarching hypothesis is that actin dynamics at the immune synapse are controlled by a regulatory complex comprised of WASP, WIP and HS1. We propose that each of these proteins plays a distinct role in regulating actin architecture, while at the same time modulating the function of the others. Totest this hypothesis, we ask whether and how complex formation is controlled (Aim 1); we will compare the contributions of individual complex components to actin responses and other aspects of T cell activation (Aim 2); and we will conduct structure-function analysis of the least well understood complex component, HS1 (Aim 3). Fig. 25 Diagram of HS1-WIP-WASP interactions. Note the modular nature of the three proteins, and the presence in WIP and HS1 of the WH2 and acidic motifs that WASP uses to drive Arp2/3-dependent actin polymerization. Known domain interactions are indicated. 1) WASP catalyzes the formation of branched actin filaments, an activity that may be regulated by WIP binding. 2) Acting together, HS1 and WIP can also catalyze new actin polymerization. 3) The HTH region of HS1 can also stabilize actin filaments, as can WIP.

16. Address Each of the NIH Review Criteria • Significance • Investigators • Innovation • Approach • Environment

17. SignificanceWithout this, all is lost. • In an era when funding is severely restricted, projects that are the most clinically and/or biologically significant are likely to be viewed most favorably. • Spell it out up front. • What diseases are involved? • How many people would benefit? • What obstacles would be overcome? • Basic knowledge? OK – but why should people care? • Make sure you hit the priorities of the chosen institute. • Consider reiterating at the end of the Approach section.

18. Investigators • Pay careful attention to your biosketch. • Most reviewers will not know who you are and make use of your biosketch to assess your credentials and accomplishments. • Use the paragraph well. Tailor it to each application. • Why are you uniquely well qualified to do this work? • Choose papers thoughtfully. • Don’t neglect the investigative team.

19. Investigator(s) • Collaborators can really strengthen an application. • Make sure that their role is commensurate with their involvement. • Co-investigator? • Collaborator? Consultant? Other significant contributor? • Paid vs. unpaid? • Add a Biosketch if it strengthens your application. • Lab members can also strengthen a team. • Include Biosketches. Mention in budget justification.

20. Environment • Why are you at a great place to do this work? • What equipment is available? • What expertise is available? • Goes back to Investigators, and beyond. • What resources are available? • Patients, specimens, etc. • This mostly gets dealt with on Resources page. • Start with a special paragraph analogous to the one on the Biosketch. • Make sure your equipment list is appropriate to the grant. • If collaborative, address contiguity or other logistics.

21. Innovation • Spell it out. • Think of this as – “Why will I succeed where others have not?” • New technology? • New question? • New way of asking an old question? • Although reviewers are attracted to novel ideas and methods, be careful. Propose good solid experimentation. • Not technically innovative? – tell them why the old way is still the best way.

22. Approach • Provide clear rationales for proposed experiments. • Don’t be overly ambitious; propose what you think you can reasonably accomplish over the tenure of the grant. • provide a tentative timeline • It is essential to provide potential alternative approaches -- not every experiment works! • Better yet – show them the result will be interesting either way. • Provide essential experimental detail, but don’t write a lab manual. • Don’t neglect statistics – more on this…

23. Approach: Preliminary Data • Ideally, some preliminary data should be provided to support each major line of investigation. • Although less is expected from new investigators, it is important to establish feasibility and some evidence of experimental competence. • Sprinkling preliminary data throughout the approach section works well. • Don’t assume reviewers will dig out your papers.

24. Address Feasibility • Don’t assume that they will trust that you can do a technique. • If its challenging, address that. • Show an example • Refer to your publications (not just a citation) • Name a skilled collaborator • Make sure that the collaborator’s role is commensurate with the work involved. • Are there alternative approaches? • Even if you have no space to articulate everything, show that you are conscious of the issue.

25. Note! The application does not begin or end with the Approach Section!

26. Abstract and Specific Aims are crucial • It is likely that only 2-3 members of a study section will read your proposal in detail, but every member votes. • What the other members know independently about your proposal may be limited to these sections. • Polish these sections when you have finished the application. • Make sure they are intelligible on their own.

27. Budget • The charge of a study section is to evaluate and rank the science of the proposals they review. But they also review the budget after the science has been scored. • Be careful to justify each item in your budget. • Propose a budget that you think is realistic for the work you propose and say why.

28. Don’t blow off the other parts • Biosketches • Letters of support • Request these early. • Be prepared to draft these yourself • Resources • Equipment • Vertebrate Animals* • Human Subjects* • Resource Sharing Plan Some of these are opportunities to sell your team or environment. Others are actually scorable and can hurt you if carelessly done.

29. Cover letter You can, and should, tell the NIH which SRGs and Institutes you think would be best able to judge your work.

30. Location, Location, Location • Center for Scientific Review (CSR) Study Sections and Study Section members are listed on the Office of Extramural Review (OER) website. • Competitors are often less dangerous than people who don’t value your topic. • Pick a Study Section that is likely to be familiar with the work you propose. • Pick an Institute whose mission best matches your research • If it’s a tie, consider paylines and other factors (e.g. mechanism). • You can request cross-listing, but usually only one institute will consider your grant. • Ask a more senior colleague for advice about where to direct your proposal. • When in doubt, call a Program Officer. They are there to help you. • Build a relationship. It may save you one day. • State your preferences in the cover letter with a sentence justifying your choice.

31. Advice for Clinical Researchers Preparation • Involve biostatistician early • Sample size will dictate budget • Budget may dictate scope of work • Other documents are not “after-thoughts” • Protection of Human Subjects • Women & Minorities • Children • Planned Enrollment • Multiple Leadership Plan • Letters of Support – allow people time to write as they may have deadlines as well • Sub-awards

32. Advice for Clinical Researchers Preparation • Have a supportive “mentor” (even for an R01) • Clarify the roles of different members of research team – i.e. collaborator vs. consultant vs. coinvestigator, etc. This has an impact if the proposal is funded • Look at scientific review committee members • Can contact SRO if you feel that additional expertise is needed in the form of ad hoc members • Also, you can request particular committee in your cover letter

33. Budget Preparation in Clinical Research • Start early! • Must work with Clinical Trial Finance Management (CTFM) • Clinical Translational Research Center (CTRC) • Contact Veronica Kain for pricing • CTRC will work with CTFM on budget • Biochemistry Core Lab Costs—David Stokes • Expect budget cuts even if you are funded

34. Submitting the Application:Sponsored Projects Office • Complete the SmartForm data and forward. • Program Announcement (PA), personnel, dates, subaward/prime institutions • Initiate certification requests. • Research Business Manager (RBM) assists with budget planning and entry. • Correct forms from subs • Sponsored Projects Officer (SPO) ensures correct forms and reviews PA. • Use Activity button to upload science. • Authorized Organizational Representative (AOR) generates 424 from Smartform. • PI has 2 days to critically review assembled proposal and make revisions; submit early to allow for this!

35. Interpreting the Summary Statement • Look for commonalities amongst reviewers • Both strengths and weaknesses • Look at scoring pattern • Low scores in one area or across the board? • Unscored/Not Discussed ≠ Hopeless

36. Common Problems Cited by Peer Reviewers • Significance • Problem not important enough • Study not likely to produce useful information • No rationale for the proposed experiments is provided • Clinical implications not clear • Not ready for prime time • Studies based on shaky hypothesis or data • Alternative hypotheses not considered • Problem more complex than applicant realizes • Problems with focus • Lack of focus in Hypotheses, Aims, or Research Plan • Research priorities not clearly defined • Project is a fishing expedition • One of the Aims is “underdeveloped”

37. Common Problems Cited by Peer Reviewers • Feasibility concerns • Insufficient preliminary data or preliminary data do not support the project’s feasibility • Overambitious research plan • Methods unsuited to objective • Concerns about the investigator • Investigator too inexperienced with proposed techniques • Too little detail to convince the reviewers that the applicant knows what he/she is doing • Concerns about productivity of investigator • Inappropriate or lack of team members with expertise

38. Common Problems Cited by Peer Reviewers • Innovation gone wrong • Proposal is technology-driven (method in search of a problem) • Insufficient attention to statistics • Important to include statistician • Sloppiness, inconsistencies, typos • Will cost you points

39. To Resubmit or Not? • Are there fatal flaws (vs. you simply didn’t make your case)? • If design has to change significantly new opportunities for weakness • Was there some enthusiasm for the project? • Did reviewers think idea was significant? • “The topic itself for children with chronic illness is innovative and of importance.” • Did reviewers think design/topic was innovative? • “The use of an “accelerated” longitudinal design is clever and seems appropriate.”

40. Planning the Resubmission • Summarize all key points of review, separated by major and minor problems • Decide how each major problem will be addressed: • Could mean adding prelim data, changing methods, enhancing team, or providing more clarity/detail/justification (e.g., making your case) • Minor problems can be addressed in one sentence of Introduction or not at all

41. Responding to Reviews in a Resubmission • Start with reminder of strengths: • “We are gratified that the reviewers found our original application “innovative and of importance,” with “detailed attention to theoretical and developmental context,” and that “the study follows in a programmatic fashion” from our earlier work. In response to the reviews, we have made the following changes:” • Categorize major areas, provide brief response to each, and refer to appropriate section of grant.

42. Responding to Reviews in a Resubmission Examples of Unscored R01 Critiques, PI Response, and New Critiques (Funded)

43. Preliminary Data • Unscored application weakness: • “Not enough preliminary data to support the study as designed” (although 1 reviewer thought prelim data was strong) • PI response: • “The reviewers were in disagreement regarding the sufficiency of the preliminary data. Since the initial application, we reran our analyses on a sample of 181 dyads. We added data related to the Child Report version of the CDMI and test-retest reliability. We also included analyses that support the validity of the CDMI and provide the foundation for several hypotheses of this proposed R01.” • Resubmission application strength: • “The CDM tool, while still undergoing further evaluation, has been developed and tested by the study team and seems to be a valuable tool for assessing several components of the decision making process.”

44. Clinical Implications • Unscored application weakness: • “The potential impact for the future clinical management of these pediatric illnesses is not clearly laid out.” • PI response: • “A new section in the Research Strategy addresses clinical implications. In addition, our team now includes nurse practitioners and physicians from the Diabetes and CF Centers, who will provide feedback regarding clinical implications.” • Resubmission application strength: • “If successful, this research could potentially inform the development of interventions and training for both adolescents and for parents.”

45. Funded Resubmission • Strengths of resubmission: • Same as before, plus new strengths were directly related to changes made in response to first review: • “The applicant was very responsive to prior critiques.” • Weaknesses of resubmission: • Only minor weaknesses • Overall design, justification, and significance were unchanged from original application, so there were not new opportunities for weakness

46. Lessons Learned as a New Investigator • Leave no question unanswered, especially regarding design and methodology • Clear and consistent writing is critical • Include timeline to show feasibility • Repetition and consistency are good • Select the right team

47. Lessons Learned as a New Investigator • Be explicit about significance and clinical implications (and everything!) • How is proposed research the logical next step based on your preliminary data and field in general? • Exactly how will data be used to inform clinical issues? Don’t take for granted that these will be obvious to the reviewer

48. New Investigator • PI is a New Investigator (NI) if he/she has not been PD/PI for a substantial NIH independent research award.  • Okay to have had R00, R03, R15, R21, F, K, etc. Basically, okay to have had anything other than an R01. • Multiple PD/PIs: If application involves more than one PI, all PIs must meet definition of NI in order for application to have NI status. • Consider carefully: Multiple PI role not ideal for new PIs

49. Early Stage Investigator • Early Stage Investigators (ESI) are NI within 10 years of completing terminal research degree or medical residency • Developed because of increasing age at first R01 and decreasing R01s from new investigators • Goal is to accelerate transition to an independent scientific career • Implemented in 2009

50. Early Stage Investigator • Special consideration during peer review: • Reviewed separately from other R01s • Reviewers focus more on proposed approach than track record and expect less preliminary data than would be provided by an established investigator. • Special consideration at time of funding: • Varies by institute: • E.g., NICHD 2010 funding strategy: • R01 payline: 13th percentile • R01 payline for NI and ESI: 15th percentile