Toolkits: Resources for Developing Service Delivery Plans(SDPs) and Program Evaluations

1. Toolkits: Resources for Developing Service Delivery Plans(SDPs) and Program Evaluations Irene Harwarth, OME Laura Gould, SERVE Annual Directors Meeting February 29, 2012

2. Presentation Objectives • Review the SDP and Evaluation Toolkits under development; • Understand the concept of alignment and logic model thinking that is fundamental to developing a MEP-specific service delivery plan and evaluation; • Participate in a group activity about measurable program outcomes.

3. Legal References for CNA and SDP

4. Legal references for Evaluation

5. Continuous Cycle – CNA – SDP - Evaluation

6. Transitioning from CNA to a Service Delivery Plan The State writes a final CNA report that documents the CNA process and findings. The findings of the CNA are used in the development of Measurable Program Outcomes and a "Needs-Based Action Plan" which provides the basis of the migrant-specific Service Delivery Plan.

7. Solution Strategies Developing "solution strategies" means thinking through the most promising solutions based on potential impact, cost and availability of resources and then discussing the specific characteristics that are necessary for the effective implementation of the solution.

8. Needs-Based Action Plan A Needs-Based Action Plan should provide a comprehensive plan for moving forward on the most important and most immediate solution strategies outlined in a Needs Assessment. (Altschuld, J.W., Witkin, B.R. From Needs Assessment to Action. (2000) Sage Publications, Thousand Oaks, CA. )

9. Required Components of a Service Delivery Plan SEAs must develop and update a written comprehensive state plan (SDP) that includes the following components: • State approved performance targets which focus on reading, mathematics, high school graduation, high school dropouts, school readiness if appropriate, and any other state identified targets for migrant children.

10. Required Components of a Service Delivery Plan (cont.) • Needs assessments that include information on the identification and assessment of the unique needs of migrant children resulting from a migratory lifestyle. • Measurable program outcomes including measurable goals and outcomes the MEP will produce statewide through specific educational or educationally-related services. • Evaluation descriptions that outline how the SEA will evaluate the MEP. (Non-Regulatory Guidance – October 2010.)

11. The SDP and PFS Priority for Services (PFS) Students: Section 1304(d) of Title I, Part C requires that states give priority to those migrant students who are most vulnerable and in need of services. Under this definition, states must give priority to migrant children: • Who are failing or who are most at risk of failing to meet the state’s academic and achievement standards, and • Whose education has been interrupted during the regular school year.

12. What else may a State include in its Service Delivery Plan? The SEA may also include the policies and procedures it will implement to address other administrative activities and program functions, such as: • Parent Involvement. • Identification and Recruitment. • Student Records.

13. What Does “Evaluation” Mean? Evaluation means systematically and methodically collecting information about a program or some aspect of a program in order to improve the program or make decisions about the merit or worth of the program. (Non-Regulatory Guidance – October 2010)

14. What are the Requirements for Evaluating the Effectiveness of the MEP? Each SEA must determine the effectiveness of the MEP through a written evaluation that measures the implementation and results of the program against the State’s performance targets, particularly for those students who are a priority for services. (Non-Regulatory Guidance – October 2010)

15. CNA, SDP and Evaluation Toolkit Resources Three toolkits will assist States to: • Develop CNAs, SDPs and Evaluations specific to State needs. • Avoid “Cookie-Cutter” or “Look Alike” CNAs, SDPs and Evaluations. • Ensure end results are unique to the State.

16. Overview of the SDP and Evaluation Toolkits • CNA Toolkit provides a process for conducting your needs assessment. • SDP Toolkit focuses on logic models and measurable program outcomes. • Evaluation Toolkit provides information on qualitative and quantitative methodologies. • All three provide resources and checklists to assist you.

17. CNA Toolkit • Step 1: Preliminary Work • Step 2: Explore “What Is” • Step 3: Gather and Analyze Data • Step 4: Make Decisions • Step 5: Transitioning to a Service Delivery Plan

18. SDP Toolkit • Key Concepts for Program Planning • Planning the Process • Addressing the Required Sections of the SDP • SDP Planning Challenges and Solutions

19. Evaluation Toolkit • Program Evaluation Plan • Collecting Meaningful Information About Quality and Effectiveness • Data Interpretation and Analysis • Organizing Evaluation Reports • Use Evaluation Results to Reassess Needs

20. Service Delivery Plan Logic Model

21. Explanation of the SDP Logic Model • Strategies should be selected to achieve the MPOs. • If they do not target the MPOs they should not be in the plan. • If the strategies are thoughtfully selected and implemented then the results should lead to achieving the MPOs.

22. What is a Measureable Program Outcome? MPOs are outcomes for the MEP to enable migrant students to achieve the state migrant student performance targets. • Focused • Detailed • Quantifiable • Clearly define what you would consider a "success" in meeting a particular need  

23. What is a Measureable Program Outcome? In addition, MPOs clearly define: • Which students will participate • What will happen in the program • What is expected to happen as a result of participation in the migrant program • In what time frame this will occur

24. Measurable Program OutcomesWhy They Matter: • Help the MEP achieve desired results at the State and local operating agency level through the provision of specific educational or educationally related services. • Help the MEP determine whether and to what degree it has met the special educational needs of migrant children that the SEA identified through the CNA. • Help migrant children achieve the state’s performance goals.

25. Setting MPOs: An Example By 2015, and each year after, the percentage of migrant students receiving the number of required hours for reading & language arts instruction will increase by 10%. The number of migrant students who attain proficiency in reading & language arts will increase by at least eight percentage points each year. In grades 3 & 8, the percentage of migrant students attaining proficient or advanced in reading & language arts was 73.2% compared to the range for all students of 86.7%. Strategy 1.1: Local programs will provide opportunities for supplemental reading & language arts instruction for migrant students beyond the regular school day.

26. MPO Example • By 2015, and each year after –[ established timeframe] • the percentage of migrant students– [specific, quantifiable, for whom] • receiving the number of required hours for reading & language arts instruction – [as a result of what MEP services] • will increase by 10%. –[quantifiable gain]

27. Participant Directions: Exercise 1MPOs on your Own • Now that we have reviewed what a Measureable Program Outcome (MPO) should entail and an example of how to create one linked to a specific identified need, strategy, and performance indicator, we are going to give you a chance to create your own MPOs. • Let’s say that your planning committee came up with the following MPO: “By the end of the 2013-2014 school year and each year thereafter, the gap between migrant students and all students will be reduced in reading/language arts in grades 3-8.” • As a group (with others at your table) please rework this MPO to be more in line with what you just learned about what a useful MPO should be? (take about 5 minutes) • What did you come up with? (discuss how people changed – 10 minutes)

28. Setting MPOs: On your Own By the end of the 2013-2014 school year and each year thereafter, the gap between migrant students and all students will be reduced in reading/language arts in grades 3-8.

29. Participant Directions: Exercise 2Setting MPOs on your Own • Now let’s say that your planning committee came up with the following need: “By the end of grade 9, only 40% of migrant students pass Algebra 1 versus 70% of non-migrant students.” • With your group, please fill in both some strategies and corresponding MPOs that will help address that need (take 5 minutes). • What did you come up with? (take 10 minutes to discuss)

30. Setting MPOs: On your Own The percentage of migrant students passing Algebra 1 by the end of grade 9 will increase by 5 percentage points each year By the end of grade 9, only 40% of migrant students pass Algebra 1 versus 70% of non-migrant students

31. For More Information Contact Irene Harwarth (202)401-3751 Irene.harwarth@ed.gov