1 / 39

Approach—Methods:

Approach—Methods:. Not an Afterthought! Blair T. Johnson CHIP | Psychology | UConn. Wow! Methods and Statistical Analysis Sections!!. After lunch?? Really???!!. My background. Assistant-Associate Professor of Psychology, Syracuse University (1988-1999)

tirzah
Download Presentation

Approach—Methods:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Approach—Methods: Not an Afterthought! Blair T. Johnson CHIP | Psychology | UConn

  2. Wow! Methods and Statistical Analysis Sections!! • After lunch?? • Really???!!

  3. My background • Assistant-Associate Professor of Psychology, Syracuse University (1988-1999) • Professor of Psychology, UCONN (1999-present) • Background: Social influence, Social Structure, Statistics (especially Meta-Analysis) • Served on numerous NIH, CDC, and UCONN grant panels • Served on IRB, two terms (2003-5; 2013-present) • Funded by NIH continuously since 1995 • 3 R01 awards from NIMH (current R01 is in year 15!) • 2 K awards, 1 from NIMH (5-years, K01) and 1 from NIAAA (1-year, K18) • This work focused on HIV prevention • Currently proposing new research on HIV prevention and on exercise and blood pressure (with Linda S. Pescatello) • New collaborations? Sure! Use my geotemporal databases to illuminate important health problems!

  4. Overview, Goals, Assumptions • Big picture: The place and importance of the Method section in your grant proposal: Not an afterthought! • 12 13 14 15 16 Tips for how to write a clear Method section that inspires confidence in your project. • Discussion throughout the session (and the next focused on the statistical analysis portion of the proposal). • Assumptions: • NIH-style grant proposals (e.g., R01, R03, R21) (though much advice generalizes to other funders). • Much of my advice generalizes to the entire grant-writing process. • Mea culpa: I’ve not yet written a perfect grant proposal.

  5. The Big Picture • What are the most important components of the grant proposal? • Summary • Resources • Budget justification • Biosketch(es) • Specific Aims • Approach (Method/Statistical Analysis) • Literature Cited • Data Safety and Monitoring Plan • Multiple PI

  6. The Big Picture • Tip 1: The Method section should flow from the preceding sections; inconsistencies increase doubt and decrease confidence in the proposal. • “Consistency is contrary to nature, contrary to life. The only completely consistent people are the dead.” (Huxley)

  7. The Big Picture Tip 3, an “Über-Tip”: Don’t Make the Reader Do Your Work For You! (Or it will hurt your score!) • Tip 2: Write in such a way that you can convince your most severe, over-worked, stress-crazed critic. • Corollary: Know the traditions and personnel of the SRG (look at what research it has supported in the recent past). • Explain in detail how you will carry out the research. • Demonstrate careful, thorough research plan. • Convey feasibility. • Describe all study procedures. • Acknowledge potential setbacks and how you will overcome them.

  8. Research Plan The research plan describes the proposed research, stating its significance and how it will be conducted. Remember, your application has two audiences: the majority of reviewers who will probably not be familiar with your techniques or field and a smaller number who will be familiar. • All reviewers are important to you because each reviewer gets one vote. • To succeed in peer review, you must win over the assigned reviewers . They act as your advocates in guiding the review panel’s discussion of your application. • Write and organize your application so the primary reviewer can readily grasp and explain what you are proposing and advocate for your application. INSIDER TIP: Appeal to the reviewers and the [program officers] by using language that stresses the significance of your proposed work. http://grants.nih.gov/grants/writing_application.htm#developPlan

  9. How Will Your Approach Be Judged? Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed? http://grants.nih.gov/grants/funding/sbirsttr_ReviewCriteria.htm

  10. Methods Section: Not an Afterthought! Tip 4: Detail matters! Corollary: Proof read your proposal! s Corollary: Carefully read all instructions in the application and application guide!

  11. SRG Reaction • “the submitted application contained a number of typos… raised concerns that lack of attention to detail may produce sloppy research…“ (paraphrased)

  12. Give an Overview, Use Headings • Tip 5: Your introductory paragraph should give an overview to the entire approach. • Basic sections: • Introduction • Justification and Feasibility • Work design/activities • Expected outcomes/results • Anticipated problems and their solutions (alternative strategies) • Tip 6: Using headings for the sections will help reviewers find the information.

  13. Use Your Specific Aims • Tip 7: Use your Specific Aims to organize the details in the Method. • A Method section should be as linear and straightforward as possible. • Organizing the Method by specific aim helps the reviewer judge whether your research is possible, and, at best, that it is highly likely to succeed.

  14. Nuance the Method Meaningfully • Tip 8: Anticipate concerns that the SRG might have about particular methods. • Corollary: Your own knowledge of the substantive domain should be high! • Write brief justifications why a certain procedure is more trusted than another. • Or, describe how you will use the data from the project to justify particular assumptions. (More on this issue in the Statistical Analysis portion!) • Showing this level of expertise helps to inspire confidence in the proposal.

  15. Must Be Justifiable and Feasible • Tip 9: Your approach must be justifiable and feasible. • Corollary: Don’t promise too much. • Corollary: Preliminary research should strongly support the approach. • Make sure your approach is consistent with current scientific literature related to the subject.

  16. Include a Detailed Timeline • Tip 10: Include a detailed timeline across the entire funded project. • Shows attention to detail, thoroughness. • Helps to show how busy you and your team will be. • Convince reviewers the proposal is reasonable with allotted time.

  17. Use Proactive and Confident Language • Tip 11: Describe your methods proactively and confidently, as though you will receive the award. • Corollary: Do not describe your project tentatively. • In other words: You and your team are poised and ready, just waiting for the grant money to begin the project. • Good example: “Our team will perform these procedures…” • Bad example: “Our team would like to perform these procedures…”

  18. Convey Excitement About YOUR PROPOSAL No one will be excited about the proposal if You Are NOT!!!

  19. Give Method Section Sufficient Space • Tip 12: Save yourself enough space to write a compelling Method section! • Corollary: Write the Methods early in the process (e.g., once Specific Aims solidify), even though most SRG members will read from front to back.

  20. Illustrations Can Save Space, Enhance Interest • Tip 13: To the extent that grant instructions permit it, use Tables and Figures to save space and enhance interest. • Corollary: Proposals can be too text-dense.

  21. Which Grant Would You Want to Read ? Ugly!!! Pretty!!!

  22. Attention to detail—thoroughness—inspires confidence in the team and the Methods proposed. • Attractive graphics imply high skill.

  23. Even an image that takes a full-page can be presented as a fraction of a page. • SRG members can zoom in if they want to see the details.

  24. Savvy Software Expertise Comes in Handy • Tip 14: Use creative illustrations. • Corollary: Graphics and illustrations lacking clarity will only frustrate a reader and decrease confidence. • Make images larger than they need to be and then resize them. • Images saved in png format generally have better scalability than other common formats (e.g., jpg, gif, bmp).

  25. USE FIGURES TO ILLUSTRATE PATHWAYS AND SPECIFIC AIMS Aim 2: TRA-1, TRA-2 interaction Aim 3: Nuclear-cytoplasmic trafficking NXF-1 and TRA-2 Aim 1: Characterize nuclear protein complex

  26. Practical Matters • Tip 15: The budget is implicit in the Method section. • Corollaries: • Make certain that the roles for all team personnel are clearly stated (and match what is said in the budget justification); highlight competence and expertise of yourself/collaborators to carry out research. • If a reviewer had doubts about your budget after reading the justification, the Method is your last chance to make an implicit pitch that your budget is appropriately scaled to the tasks that your team will perform. • The Method should implicitly sell what a great value the research would be to conduct.

  27. But Your Method Is about Science, Not Budget! • Tip 16: Leave arguments about budget per se in the Budget Justification section. • Your Method section should mention money only in connection to practical matters (e.g., incentive payments to participants). • But the details in the Method should make clear why particular personnel and their time are needed for the success of the project.

  28. Wow! Statistical Analysis Section!! • After the coffee break! • Really!!

  29. Approach: Statistical Analysis—Not an Afterthought! Blair T. Johnson CHIP | Psychology | UConn

  30. Goal of the Statistical Analysis Section • State how the Specific Aims will be addressed in your statistical analyses. • Makes a case that the study has a good chance to detect the proposed effects (viz., “is sufficiently powered”). • Assumes sampling individual-level observations. • In meta-analysis: Examine all available qualifying observations. • The concern is more with whether there are enough studies to permit tests of the hypotheses. • Power analysis is less well developed here. • NIH now requires a power analysis for most proposed research.

  31. Tips for a Successful Analysis Section • Examine multiple models of analysis sections from high-quality research similar to your own. • Justify the analysis method as the best in practice. • Early consultation with a statistician is recommended, but there are many statistical software programs available (ensure statistical analysis plan is sound). • For power analysis, select effect size standards carefully (ideally from a meta-analysis of relevant trials). If you select too large an effect size, you will be underpowered to detect the effect!

  32. Power Analysis • In order to minimize the reporting of false-negative data, a power analysis should be performed for sample size determination. Power is the capability of a study to detect a difference if the difference really exists. A type II error occurs when a true difference exists between study populations but there are insufficient numbers of subjects to detect this difference.

  33. Power/Sample Size Example *Required sample size to address study aims, if a medium effect size is assumed.

  34. G*Power 3.1.9.2

  35. More Tips for a Successful Analysis Section • Generally, statistical analysis sections should use conservative estimates of effect size to ensure that study is well-enough powered. SRG members are likely to be savvy about standards and easily suspicious, so it pays to sweat the details. • Discuss how you will use sensitivity analyses to examine the robustness of your main analyses.

  36. Critique? “To establish an estimate of power and to project proper sample size, a previous study quantifying the dependent variable was used using GPowersoftware 8. For a 2-sided within-group design at a 0.05 alpha level and desired power level of 0.95, the estimated sample size was 11 for each gender. We will enroll up to 14 subjects for each group in order to increase statistical power as well as take into account potential drop outs. Dependent T-tests will be used to assess statistically significant differences between conditions.”

  37. Example 2 With n=400, alpha2 = .05, and a medium effect size (.30) the power will be >.99 for analyses of variance and .96 for zero order correlations120. The power for a regression analysis that includes 11 variables (i.e. sex, ethnicity, positive symptoms, negative symptoms, aggression, impulsivity, depression, premorbid adjustment, gene marker, family history and substance abuse) with n=400, alpha=.05 and a medium effect size (R2=.10) will be greater than .90. We do not anticipate that all 11 variables will contribute significantly to the model. With 6 variables, a more likely model, the power will be > .90. For the exploratory regression analyses conducted within the group of attempters (n=200) the power will be at least .80 to detect a medium effect size (R2=.10). For correlational studies among the genetic and biochemical measures (approximate n=100) the power will be .80 for a medium effect size (R2=.10) and .95 for a larger medium effect size (R2=.25). The increased sample size will now provide the power necessary to consider attempters and nonattempters separately. Analyses examining single attempters, multiple attempters and nonattempters (approximate sample sizes: 120, 80, 200) will still maintain adequate power (power>.70). Grant application by Jill M. Harkavy-Friedman, PhD

  38. Good Luck! If at first you don’t succeed – Revise and resubmit

More Related