Grant Writing Workshop for Early-Career Investigators

1. Grant Writing Workshop for Early-Career Investigators Allen Ruby, Ph.D. Associate Commissioner for Policy and Systems National Center for Education Research Amy Sussman, Ph.D. Program Officer National Center for Special Education Research

2. Today’s Purpose To provide advice to early career researchers on submitting funding applications to the Institute of Education Sciences To review IES’ FY 2013 requests for applications and their requirements

3. The Big Question:Can I be the PI on an IES Grant? YES, if . . . • Your background supports your role as PI • A strong team can supplement your background • You write clearly and address the reviewers • You address the objectives and student outcomes of interest for IES’ grant funded research • You apply to the right grant program • Education Research Grants (84.305A) and Special Education Research Grants (84.324A) • Other research grant programs • You address the key requirements of the Research Narrative • You address the submission requirements

4. Your Challenge Your challenge is to convince reviewers that you and your team have the knowledge, skills, and experience to implement well what you have proposed.

5. Background • Receiving a grant is a process that begins before the submission of your application • Review your past experience as a researcher. Can you argue that you have the knowledge/skills to be a PI • Substantive • Methodological • Project management • Productivity • You will need to document this experience in your application more than an experienced researcher would

6. Examples of a Strong Background • Built a record in a substantive topic and/or methodology as a graduate student, as a postdoc, and in your present position • Published on own or with well-regarded experienced researchers in substantive area you propose to do future work in well-regarded journals • Published from dissertation and other projects • Supports your future work in this topic • Held major responsibilities within a project • Provides experience in carrying out project • New projects can be based on finding of old ones • Extensive work with a secondary data set • Provides a foundation for future research using this data

7. Building Your Research Record • Run smaller projects (e.g., internal grants, small external grants) • Take on roles in larger external grants • Sometimes you have to do work not directly in your immediate interest • Sometimes you can carve out space under another PI’s project to do your work of interest • Usually related to the project’s goals • Make sure you get lead/sole authorship

8. Building Research Record (cont.) • In either case, take on roles that will provide you with the experience to lead a future project • Co-I • Co-PI • Project manager • Data manager • Build expertise, e.g. substantive area, specific design, analytical method, data set • Build links to school/district/state education leadership • Publish from all these types of projects • Provide useful information to practitioners

9. Building a Good Team • Review the expertise needed to carry out your proposed project • Review your expertise • Recruit people with well-regarded specific expertise needed by project, e.g.: • Statistician/econometrician or substantive expert for a secondary data analysis • Survey developer or implementor for a data collection • Curriculum developer or professional development expert for a project to develop an intervention • Experimental design for an evaluation of an intervention • Psychometrician or substantive expert for a measurement project • Project manager

10. Building a Team: Synthetic Reviewer Comment Given that the PI has not run a large federal grant and does not appear to have formal training in reading I would recommend that senior researchers who have expertise in reading and have received funding in the past be added to the grant.

11. Building a Team (cont.) • Be aware that the peer reviewers want to be assured that early career researchers have access to experienced researchers • Project management • Substantive and methodological areas • Including areas that the early career researcher has expertise in

12. Building a Team: Synthetic Reviewer Comment The lead investigator is junior and has not run a study of this scope before. However, this is only a moderate weakness given his prior experiences with data collection and the excellent supervision and support that he will receive.

13. Building a Team (cont.) • Including experienced researchers can be done in various ways • Co-PI • Co-I • Advisory Panel • Consultants • They should be on for enough time to be seen as able to make a contribution to project • They should have clear roles, duties, and periodic input • Especially true for advisory panels and consultants

14. Building a Team: Synthetic Reviewer Comment The members of the advisory panel have been involved in the design and conduct of many experiments and will be of great help to the Project Director in the development of the interview protocol, in the analysis phase of the project, and in reviewing papers stemming from the analyses.

15. Write Clearly • Reviewers often complain about lack of clarity in applications • The significance of the project is written about in too general of terms • There is a lack of detail on a key aspect of the work, e.g., the factor or intervention being studied, the cycle for developing an intervention, or the description of the data analysis • Reviewers have less information about you to judge your abilities and so may be less willing to give you the benefit of the doubt when writing is unclear or lacks detail

16. Write Clearly: Synthetic Reviewer Comment No information is given regarding the actual interventions to be used with the at-risk students in the control and experimental conditions. This makes it impossible to judge the potential impact of the intervention on the reading outcome of at-risk children.

17. Write Clearly: Synthetic Reviewer Comment The data analysis plan is unclear. What kind of scores from the measures will be used in what specific models to address which specific research questions. Many different analyses are mentioned but their tie-in with specific research questions is unclear. In the regression discontinuity model that is provided how are variables centered to provide the interpretation of the coefficients as provided? What is the sample size that will be used? How will clustering be handled? Why are grade level and time both in the model? More information is needed to help understand the analysis plan.

18. Write Clearly (cont.) • Maintain consistency throughout application • Ensure that sections support one another • The research plan addresses the original research questions that were justified as significant • Write for both generalist and specialist • Avoid use of jargon and assumptions of knowledge • Have both an expert and non-expert do a read through

19. Write Clearly: Put the Punch Line Upfront • Opening paragraph sets the scene for the reviewers • Identifies the significance of the work to be done and what actually will be done • Reviewers use it to organize information from rest of application • You can lose your reviewers right off with an unclear opening

20. Address the Reviewers • Reviewers focus on the Research Narrative (more on this later) • Reviewers include both specialists in your area and generalists • Initial reviews are done by a substantive reviewers and a methodologist • Panel has expert in every component of your study • Show personnel can do the work • For resubmissions, address the previous reviews

21. Advice From an Early Career PI • Be a co-PI (or key personnel) on a previous project • Highlight your management role from this project • Can be small internal grant, state or foundation grant, or federal grant • Have personnel with 'senior expertise' that you don't have • Add Co-PIs who have received federal grants and have expertise in critical elements for the project and/or • Have an advisory board with senior people, a clearly defined role, and access throughout the year not just at the formal meetings • Reviewer feedback suggested that although I was junior, I had built a strong team that could support me

22. Advice From an Early Career PI (cont.) • Extend off of your prior research work • Built directly off of dissertation work that had been published in a good peer-reviewed journal • Continued this work with a small internal grant from own institution • IES Application was based on data and results from dissertation and the internal grant study • Try again • Most of the reviewer feedback was very helpful in strengthening the second application which was funded

23. Applying for an IES Grant • Address the objectives and student outcomes of interest for IES’ grant funded research • Apply to the right grant program • Education Research Grants (84.305A) and Special Education Research Grants (84.324A) • Other research grant programs • Address the key requirements of the Research Narrative • Address the submission requirements

24. IES’ Organizational Structure Office of the Director National Board for Education Sciences Standards & Review Office National Center for Education Statistics National Center for Education Evaluation National Center for Education Research National Center for Special Education Research

25. Overall Research Objectives For Grant-Funded Research Develop or identify education interventions (practices, programs, policies, and approaches) that enhance academic achievement and that can be widely deployed Identify what does not work and thereby encourage innovation and further research Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness

26. Final Outcomes of Interest are for Students (Student Outcomes) Birth through Preschool School readiness for both Centers Developmental outcomes for infants and toddlers with or at-risk for disabilities Kindergarten through Grade 12 Academic outcomes in reading, writing, math and science Behaviors, interactions, and social skills that support learning in school High school graduation Functional outcomes that improve educational results, transitions to employment, independent living, and postsecondary education for students with disabilities

27. Student Outcomes (cont.) Postsecondary Enrollment, persistence, completion Achievement in gateway math & science courses Achievement in introductory composition courses Adult Education Reading, writing, and math for adult basic and secondary education and English language learners

28. Applying for an IES Grant (cont.) • Apply to the right grant program • Education Research Grants (84.305A) and Special Education Research Grants (84.324A) • Other research grant programs • Address the key requirements of the Research Narrative • Address the submission requirements

29. Primary Research Grant Programs Education Research Grant Programs (84.305A) Special Education Research Grant Programs (84.324A) These grant programs are organized by research topic and research goal

30. Education Research Topics (84.305A) • Cognition and Student Learning • Early Learning Programs and Policies • Education Technology • Effective Teachers & Effective Teaching • English Learners • Improving Education Systems: Policies, Organization, Management, and Leadership • Mathematics and Science Education • Postsecondary and Adult Education • Reading and Writing • Social and Behavioral Context for Academic Learning

31. Special Education Research Topics (84.324A) Autism Spectrum Disorders Cognition and Student Learning in Special Education Early Intervention and Early Learning in Special Education Families with Children with Disabilities Mathematics and Science Education Professional Development for Teachers and Related Service Providers Reading, Writing, and Language Development Social and Behavioral Outcomes to Support Learning Special Education Policy, Finance, and Systems Technology for Special Education Transition Outcomes for Secondary Students with Disabilities

32. FY2013 Research Goals Exploration Development & Innovation Efficacy and Replication Effectiveness Measurement

33. Exploration Goal • Explore associations between education outcomes and malleable factors • Identify factors and conditions that may mediate or moderate the relations between malleable factors and student outcomes • Possible methodological approaches include: • Analyze secondary data • Collect and analyze primary data • Complete a meta-analysis

34. Development & Innovation Goal Development process must be iterative! • AND collect data on its feasibility and usability in actual education settings • AND collect pilot data on student outcomes. Develop an innovative intervention (e.g., curriculum, instructional approach, education practice, program, or policy) OR improve an existing education intervention

35. Efficacy & Replication Goal Evaluate whether or not a fully developed intervention is efficacious under limited or ideal conditions OR Gather follow-up data examining the longer term effects of an intervention with demonstrated efficacy OR Replicate an efficacious intervention varying the original conditions

36. Effectiveness Goal Evaluate whether a fully developed intervention that has evidence of efficacy is effective when implemented under routine practice through an independent evaluation Prior to submitting an effectiveness proposal, at least two efficacy studies of the intervention with beneficial and practical impacts on student outcomes must have been completed Follow-up projects to determine longer-term impacts of intervention that has been found effective can also be proposed

37. Measurement Goal Development of new assessments or refinement of existing assessments, and the validation of these assessments OR Validation of existing assessments for specific purposes, contexts and populations

38. Award Maximums (84.305A & 84.324A)

39. Applying for an IES Grant (cont.) • Apply to the right grant program • Education Research Grants (84.305A) and Special Education Research Grants (84.324A) • Other research grant programs • Address the key requirements of the Research Narrative • Address the submission requirements

40. Early Career Program • Research Training Program in Special Education: Early Career Development and Mentoring (CFDA #84.324B) • New competition for FY2013 • Considered a research training grant • Requires research project

41. Early Career Program (cont.) • Eligibility for Principal Investigator • Focus of research on infants, toddlers, children, or youth with or at risk for disabilities • Need for additional research training • Completed doctoral degree or postdoctoral training within 3 years of the application due date • Tenure-track position at IHE • No previous IES award as PI or co-PI

42. Early Career Program (cont.) • Components of Early Career Program • Research Plan • Plan of research that corresponds to NCSER topic and goal structure • Career Development Plan • Mentoring plan (primary mentor, possible co-mentors) • Additional training plan (e.g., workshops, courses, summer institutes) • Explicitly integrated with research plan

43. Statistical and Research Methodology in Education (84.305D) Research projects intended to expand and improve the methodological and statistical tools available for education researchers These tools will be used to improve the design of research studies, analysis of research data, and interpretation of research findings

44. Evaluation of State and Local Education Programs and Policies (84.305E) Support for rigorous evaluations of education programs or policies that are paid for and implemented by State or local education agencies Evaluations are to determine both the overall impact of the programs/policies and the impact across a variety of conditions

45. Researcher-Practitioner Partnerships in Education Research (84.305H) Intended to support the partnering of researchers with State and local education agencies in the development of joint research projects

46. Accelerating the Academic Achievement of Students with Learning Disabilities Research Initiative (84.324D) To develop and evaluate interventions (e.g., instructional approaches, curricula, technology) to accelerate the reading and math achievement of students with or at risk for reading and math disabilities in grades 3 through 8 Will create a tightly linked network of researchers across a variety of disciplines who will work collaboratively to address the problem

47. Award Maximums (84.305B-H & 84.324B-D)

48. Applying for an IES Grant (cont.) Address the key requirements of the Research Narrative Address the submission requirements

49. Research Narrative • Key portion of your application • Comprised of four sections: • Significance • Research Plan • Personnel • Resources

50. Significance Section • Describe the overall project and the Research Questions you intend to answer • Provide a compelling rationale for the project • Varies by grant program • For 84.305A and 84.324A varies by research goal