Overview of Education Research & Development Centers Program (84.305C)

1. Overview of Education Research & Development Centers Program (84.305C) Rebecca Kang McGill-Wilkinson, Ph.D. Institute of Education Sciences

2. Outline • Overview of IES • Research & Development Centers • Developmental Education Assessment & Instruction • Knowledge Utilization • General requirements • Deadlines and notification

3. Legislative Mission of IES Describe the condition and progress of education in the United States Identify education practices that improve academic achievement and access to education opportunities Evaluate the effectiveness of Federal and other education programs

4. Organizational Structure of IES Office of the Director National Board for Education Sciences Standards & Review Office National Center for Education Evaluation National Center for Education Research National Center for Education Statistics National Center for Special Education Research

5. Missions of the Research Centers • NCER • supports rigorous research that addresses the nation’s most pressing education needs, from early childhood to adult education • NCSER • sponsors a rigorous and comprehensive program of special education research designed to expand the knowledge and understanding of infants, toddlers, and students with or at risk for disabilities from birth through high school

6. IES Grant Programs: Research Objectives • Develop or identify education interventions (practices, programs, policies, and approaches) • that enhance academic achievement • that can be widely deployed • Identify what does not work and thereby encourage innovation and further research • Understand the processes that underlie the effectiveness of education interventions and the variation in their effectiveness

7. FY 2014 Research & Research Training Grant Programs • Education Research Grants Program (84.305A) • Education Research & Development Centers (84.305C) • Statistical & Research Methodology in Education (84.305D) • Statistical & Research Methodology Grants • Early Career Statistical & Research Methodology Grants • Partnerships & Collaborations Focused on Problems of Practice & Policy (84.305H) • Researcher-Practitioner Partnerships in Education Research • Continuous Improvement Research in Education • Evaluation of State & Local Education Programs & Policies • Research Training Grant Programs in the Education Sciences(84.305B) • Predoctoral Interdisciplinary Research Training • Methods Training for Education Research • Training in Education Research Use & Practice

8. NCER Investments by Competition

9. Research & Development Centers: Purpose • Contribute to solving U.S. education problems, improve the education system, and increase student achievement through • Research • Development • Evaluation • National leadership

10. Research & Development Centers: Activities • Focused research program • Contribute to solution of specific education problem within the Center’s topic • Generate new knowledge and theories • Supplemental studies • Research on supplemental questions within the Center’s topic • Leadership and outreach • To researchers, practitioners, and policymakers • Advance the field and practice through evidence-based research, practice, and policy

11. Center Partnerships • Institute encourages partnerships with stakeholder groups to increase • Relevance of work • Accessibility of publications • Usability of findings

12. National Research & Development Center on Developmental Education Assessment & Instruction

13. Purpose • Research • Document current developmental education practices • Identify promising programs • Support innovation • Evaluate programs that serve large numbers of students and/or that can be expanded • Assess scalability of programs found to benefit students • Leadership and outreach • Bring together policymakers, practitioners, and researchers • Assist states and postsecondary institutions to bring effective models of developmental education to scale

14. Background: Research • Large numbers of students beginning postsecondary education – especially in community colleges and open-access 4-year institutions – are placed into developmental (remedial) reading, writing, or mathematics courses before they can accumulate credits • Many students are required to take multiple developmental education courses

15. Background: Research • Research suggests tests and other practices used to place students in these courses may fail to predict who will benefit from developmental education • Studies that track progress through coursework show mixed results • Students may not complete coursework • Students assigned to developmental education may be less likely to graduate

16. Background: Policy Concerns • Decline in support for developmental education • In some corners, seen as more of a hindrance to students than a support • Expenditures on developmental education • Target for cost savings • More students may be assigned to developmental education than need it • Placement and instruction need to be revised or replaced

17. Background: Ongoing Work to Improve Developmental Education • Change placement procedures • Replacing standardized test with other measures • E.g., high school transcript, college-readiness indicators • Change cut scores for tests • Identify high school students’ need for developmental education and provide remediation • During high school rather than college • During a “bridge program” before college • Providing additional supports during college • E.g., learning communities

18. Background: Ongoing Work to Fundamentally Change Developmental Education • Acceleration • Compress 1-year’s worth of material into a shorter period • Modularization • Students complete a module of study at their own pace • Students who start with more knowledge can finish sooner • Mainstreaming • Development education students take college-level courses with additional support • Replacement • Replace conventional developmental math education sequence with one on quantitative reasoning and statistics

19. The Center’s Focused Program of Research • One descriptive study • To document current developmental education practices (assessment and instruction) in use by community colleges and open-access institutions • Two evaluation studies • To determine if specific assessments or instructional practices lead to improved student outcomes in reading, writing, math, English language proficiency, or any combination of these

20. The Descriptive Study: Research Questions • What are the assessment tools and practices used to assess reading, writing, and math skills of incoming students and place them in developmental education (versus college-level) courses? • Why do they use these tools and practices? • What are the major strategies used to teach developmental education? • Are different assessment and instructional approaches used for nontraditional students? • Are the assessment and instructional approaches aligned with the Common Core State Standards? • Applicants may expand or add additional research questions (and should provide justification for doing so)

21. The Descriptive Study: Research Approach • Institutional survey • Stronger if nationally representative of community colleges and open-access institutions • If not national, stronger if focus on State or postsecondary institution in which evaluation studies will take place • Qualitative interviews • Secondary data analysis • Combination of the above

22. The Descriptive Study: Methodological Requirements Research questions Population from which study sample will be selected Measures Data analysis plan Timeline

23. Evaluation Study 1: Assessment & Placement • Possible Research Questions • How can institutions improve on current assessmentsand placement practices? • What are the effects of alternative assessment and placement strategies on students’ overall academic performance, persistence, and progress? • You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation

24. Evaluation Study 2: Instruction • Possible Research Questions • How can institutions prepare students for college-level courses in the shortest amount of time? • What are the impacts of different instructional approaches on student learning, overall academic performance, persistence, and progress? • You may propose expanded or modified research questions as long as they meet the purpose and requirements of the evaluation

25. Evaluation Studies: Methodological Requirements • Research questions • Sample and setting • Appropriate research design (meets WWC standards) • Power analysis • Measures • Proximal and distal outcomes • Including student outcomes in reading, writing, math, English language proficiency, or some combination of these • Fidelity of implementation • Comparison group practices

26. Evaluation Studies: Methodological Requirements • Moderators or mediators • Plan to analyze implementation fidelity and comparison group practices • Data analysis plan • Overall impacts • Subgroup impacts (e.g., nontraditional students and students taking more vs. fewer developmental edcourses) • Issues of scalability • Including cost-benefits of the programs evaluation • Timeline

27. Supplemental Studies • May conduct smaller, supplemental research projects • Usually can be completed within 9 to 12 months • May include exploratory research, piloting innovations, or further development of innovations • Center will work cooperatively with the Institute to select and design supplemental studies • Must provide two examples of supplemental studies in application • Short rationale and brief description of approach (2 to 3 paragraphs)

28. Leadership & Outreach • The Center will work with the Institute on its leadership and outreach activities. These may include: • Convening policymakers, practitioners, and researchers • Assisting efforts by States, colleges, and universities to bring effective models to scale • The Institute encourages a Center website, webinars, podcasts, and other innovative uses of technology • You should describe at least two examples of activities in application

29. Significance of Center • Significance section at start of Research Narrative • Discuss the significance of your proposed Center • Your vision of the Center • What problems it is intended to address • How your set of research and leadership activities will work together to help students in developmental ed • How the research will be relevant to community colleges and open-access 4-year institutions • Describe active partnerships with States or postsecondary systems and their role in the work

30. Maximum Awards 5 years $10,000,000 Maximum of one award Focused program of research must comprise at least 75% of the total budget

31. National Research & Development Center on Knowledge Utilization

32. Purpose • Research • Develop tools for observing and measuring research use in schools • Identify conditions under which practitioners use research and factors that promote or inhibit research use in schools • Identify or evaluate strategies that researchers can adopt to make their work more meaningful to and impactful on education practice • Leadership and outreach • Effective means of knowledge transfer to improve school performance and student outcomes

33. Background • Accumulation of high-quality research and evaluation on impact of education policies and interventions • Concern that this work is not used by practitioners • To adopt interventions with beneficial effects on students • To discard interventions with null or adverse effects

34. Background: Potential Causes • Misalignment between research questions (e.g., “Does it work?”) and practitioners’ concerns (e.g., “How do I use it with my students?”) • Misunderstanding of social context of practitioners • Research use is multi-level phenomenon • Need new ways of working with practitioners • Partnerships • Bidirectional model of relationship • Intermediary organizations

35. The Center’s Focused Program of Research • One measurement study • Develop and validate tools for observing and measuring research use in schools and school districts • Two descriptive studies, or one descriptive study + one evaluation/pilot study • Identify the conditions under which research is used and factors that promote research/inhibit research use in schools and districts • Identify or evaluate the skills and strategies researchers can adopt to make their work more meaningful and have a greater impact on practice

36. The Measurement Study: Research Questions What are the most reliable and valid indicators of school staff using education research in their decision making? What are the most reliable and valid approaches to capture variation in research use across schools, school districts, and over time? Applicants may expand or add additional research questions (and should provide justification for doing so)

37. The Measurement Study: Methodological Requirements Conceptual framework to provide basis for assessment, intended population, and validation Sample and setting Characteristics, size, analytic adequacy of sample Methods for developing and validating Methods for gathering psychometric evidence Appropriateness for end users Need for adaptation for different contexts and purposes Timeline

38. Descriptive Study 1: Research Questions What role do State and local education agencies play in making research available to school administrators, faculty, and staff? What role do school leaders play? How do education agencies, school leaders, and teachers obtain and process scientific information? How do social networks within schools and school districts appear to support or hinder research use? What are the major incentives or disincentives for practitioners to better understand and apply education research? What contextual factors seem to affect the use of research in schools and school districts? Applicants may expand or add additional research questions (and should provide justification for doing so)

39. Descriptive Study 2: Research Questions Are partnerships between researchers and practitioners associated with greater use? Are intermediaries associated with greater use? What communication strategies or technological innovations are associated with greater use? Applicants may expand or add additional research questions (and should provide justification for doing so)

40. Optional Evaluation or Pilot Study In lieu of second descriptive study Examine efficacy of researcher practices on research use OR Rigorous evaluation of impact of particular practice OR Pilot study to show promise of particular practice

41. Descriptive Studies: Methodological Requirements • Research questions • Population from which study sample will be selected • Institute encourages you to select two or more research sites • Measures • Data analysis plan • Timeline

42. Optional Evaluation/Pilot Study: Methodological Requirements • Research questions • Sample and setting • Appropriate research design • if doing an evaluation, the design should meet WWC standards • Power analysis • Measures • Proximal and distal outcomes • Fidelity of implementation • Comparison group practices

43. Optional Evaluation/Pilot Study: Methodological Requirements Moderators or mediators Plan to analyze implementation fidelity and comparison group practices Data analysis plan Timeline

44. Leadership & Outreach Innovative approaches to fostering dialogue among practitioners and researchers May take advantage of annual conferences Institute encourages websites Institute is open to learning how other applied social sciences encourage research use among practitioners Describe at least two examples of activities in application

45. Supplemental Studies • May conduct smaller, supplemental research projects • Usually can be completed within 9 to 12 months • Center will work cooperatively with the Institute to select and design supplemental studies • Must provide two examples of supplemental studies in application • Short rationale and brief description of approach (2 to 3 paragraphs)

46. Significance of Center • Significance section at start of Research Narrative • Discuss the significance of your proposed Center • Your vision of the Center • What problems it is intended to address • How your set of research and leadership activities will work together to promote the use of scientific education research in schools and districts • Describe how you will engage researchers and practitioners in the work and communication of findings

47. Maximum Awards 5 years $5,000,000 Maximum of one award Focused program of research must comprise at least 75% of the total budget

48. General Requirements

49. Additional Sections of the Research Narrative for Both Centers • In the 35-page Research Narrative, discuss… • Significance • Research Plan for the Focused Program of Research (including supplemental studies) • Other Center Activities (Leadership and Outreach) • Management and Institutional Resources • Personnel • We’ve discussed the first three and now let’s move to the last two

50. General Requirements: Management & Institutional Resources Describe plans and procedures for overall management of the Center and its activities If plan is to work in schools or other education delivery settings in first year, must include letter(s) of agreement from education organization(s) If developer/distributer of program or intervention is part of Center, describe role and how objectivity will be maintained