1 / 31

Evaluating the Impact of MEP Programs and Services

Evaluating the Impact of MEP Programs and Services. Presented by: John Wight Program Manager Georgia Department of Education Migrant Education Program & Omar Lopez- Nunez State Research & Development Coordinator Georgia Department of Education Migrant Education program.

stephanie-snider
Download Presentation

Evaluating the Impact of MEP Programs and Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating the Impact of MEP Programs and Services Presented by: John Wight Program Manager Georgia Department of Education Migrant Education Program & Omar Lopez-Nunez State Research & Development Coordinator Georgia Department of Education Migrant Education program Title I Conference June 19-20, 2014

  2. Evaluating the Impact of MEP Programs and Services Overview: • Evaluation Definition & Background Information • Statewide Service Delivery Plan Evaluation of Services • Current Impact of MEP Program and Services as Measured by Evaluations

  3. Program Purpose The purpose of the MEP in Georgia (and the United States) is to ensure that migrant children fully benefit from the same free public education provided to all children and that the unmet education-related needs resulting from their migrant lifestyle are met.

  4. Legal Reference for Program Evaluation

  5. Legal Reference for Program Evaluation Under Title I, Part C, Education of Migratory Children of the Elementary and Secondary Education Act, State educational agencies (SEA) must deliver and evaluate Migrant Education Program (MEP)-funded services to migratory children based on a State plan that reflects the results of a current statewide comprehensive needs assessment.

  6. Georgia MEP Program Evaluation Section 1304(c)(5) of the Elementary and Secondary Education Act (ESEA) requires states to conduct a program evaluation for the Migrant Education Program. The purpose of conducting an evaluation of the Georgia Migrant Education Program is to examine program effectiveness and results of implemented program activities.

  7. Georgia MEP Program Evaluation • The Georgia MEP will conduct a statewide summary evaluation of all program projects and services based on the guidance and suggestions delineated in the U.S. Department of Education’s MEP Program Evaluation Toolkit at the end of every academic year. • In order to prepare for, conduct, and report on a statewide evaluation plan of the Georgia Title I, Part C Migrant Education Program, the Georgia MEP will:

  8. Georgia MEP Program Evaluation • Conduct a review of current and existing data, data sources and related reports generated by the state, regional, and or local migrant projects; • Disaggregate statewide assessment data and compare the progress of migrant students with non-migrant students; disaggregate statewide assessment data and compare migrant PFS students with non-PFS and non-migrant students; • Review and compare the performance of migrant PFS students and migrant non-PFS students within the national Government Performance and Results Act (GPRA) measures; • Conduct on-site visits at the local project levels for the purpose of conducting implementation plan observations and gathering additional facts and information relative to project plans and implementation fidelity; and • Analyze the information gathered and create a written evaluation incorporating implications and recommendations for overall program improvement, marking the close of the Continuous Improvement Cycle (CIC) for the Georgia MEP.

  9. MEP Evaluation Cycle: An Ongoing Process Georgia Continuous Improvement Cycle (GCIC):

  10. Georgia MEP Continuous Improvement Cycle (GCIC) The strategies for delivery of services have been determined by setting a three-step project planning process that every LEA must follow in order to ensure fidelity in the Continuous Improvement Cycle (CIC) of the Georgia MEP. The three steps in the project planning process for LEAs involve: • Submission of a CNA profile that captures the needs of the migrant population at the LEA level; • Submission of implementation plan(s) in order to establish academic support services to be provided with projected measurable outcome(s); • Observations of implementation plans and services to monitor fidelity of implementation (GaDOE MEP and LEA staff); completion of surveys by SSPs to monitor fidelity of implementation; and • Completion of implementation plan evaluation(s) for each implementation plan submitted at the end of project cycle in order to validate actual measurable outcome(s) as projected on original implementation plan(s) submitted.

  11. Georgia MEP Continuous Improvement Cycle (GCIC) - Continued

  12. Program Evaluation Defined Evaluation is the systematic application of scientific methods to assess the design, implementation, improvement or outcomes of a program (Rossi & Freeman, 1993; Short, Hennessy, & Campbell, 1996).

  13. Evaluation Consideration Effective evaluation is not an "event" that occurs at the end of a project, but is an ongoing process which helps decision makers better understand the project; how it is impacting participants, partner agencies and the community; and how it is being influenced/impacted by both internal and external factors. W.K. Kellogg Foundation Evaluation Handbook, p. 3

  14. Why do we conduct evaluations? • Establish model programs and best practices by providing feedback about what worked and what failed. • Tool of good management and quality improvement - gain insight into effective strategies on how to improve performance. • Measures impact the program is making.

  15. 2013 Statewide Service Delivery Plan (SDP) • Continuous Improvement Cycle (CIC) for the Georgia MEP • Service Delivery • Evaluation 2013 Statewide CNA SDP Report

  16. Service Delivery Plan (SDP) – Overview • As required under Section 1306 of the reauthorized ESEA, the Georgia Migrant Education Program (Georgia MEP) has developed a statewide Service Delivery Plan (SDP) to be implemented during the 2013-2016 school years. • This state plan is a current and comprehensive plan for how the services provided by the Georgia MEP and local educational agencies (LEAs) are to be delivered in order to meet the needs of the migrant children and youth throughout the state.

  17. Service Delivery Plan (SDP) – Purpose (Continued) • Design Georgia programs to help migrant children and youth overcome educational disruption, cultural and language barriers, social isolation, various health-related problems, and other factors that inhibit their ability to do well in school, and to prepare them to make a successful transition to postsecondary education or employment; and • Ensure that migrant children and youth benefit from the state of Georgia and local systemic reforms.

  18. MEP Service Delivery Plan Goals Goal 1: Migrant students …will improve their writing proficiency ……. Goal 2: Migrant students … will improve their math proficiency …. Goal 3: The Georgia Migrant Education Program will improve school readiness… Goal 4: The MEP …will continue to support … OSY/DO at the district level….to foster English language acquisition, health and other relevant supplemental services… Goal 5: Migrant students…will continue to meet and/or exceed their proficiency in reading…. Goal 6: Georgia MEP SSP staff …will improve their professional competencieswhen working with migrant participants for short periods of time …

  19. Performance Targets • ESEA Flexibility Waiver • Migrant is not a subgroup • College and Career Ready Performance Index (CCRPI) • Migrant is not a subgroup • State performance targets in elementary, middle and high school • Migrant is not a subgroup

  20. Migrant Students Performance Targets and Results • The SDP will set the state performance targets for migrant students on par with the performance targets set for all students in CRCT Reading, English/Language Arts and Math (elementary and middle school level), EOCT in 9th Grade Literature, American Literature, Mathematics I and Mathematics II (high school level) and graduation rate performance targets in Georgia. • So, as the all-student population makes progress (as established in the performance targets), so will migrant students statewide. • The State MEP and LEA MEPs will analyze the progress of migrant students as compared with non-migrant, and migrant Priority for Service (PFS) students as compared with migrant non-PFS and non-migrant students on local and state formative and summative assessments.

  21. Sample CRCT Assessment Migrant vs. Non-Migrant

  22. Migrant Students Performance Targets and Results • State-level academic performance data for migrant students is used by the Georgia MEP to ensure migrant student academic progress follows that of the all students’ performance targets in the state. • These data are also used by the Georgia MEP to develop program policy and to target specific migrant education projects and interventions that will increase the academic achievement and success of migrant children and youth statewide.

  23. Current Impact of MEP Program and Services as Measured by Evaluations from 2012-2013 School Readiness and Preschool K-12 Enrolled Children Out of School Youth

  24. Measuring Fidelity 2013-2014 • Completed by SSPs and Tutors After Observation • Planning and Delivery • Student Engagement

  25. Measuring Fidelity 2013-2014 Provided by SSPs and Tutors After Observation • Application of Professional Development: • PDNow!, RESA, school level • Conference or workshop • MEP staff • Identifying the Instructional Strategy • Guided reading, direct instruction, modeling • Manipulatives, visual aids, hands on activities • Teacher guided questioning

  26. Reporting Evaluation Results • The level and scope of content depends on to whom the report is intended (OME, State, LEAs staff, Parent Groups – PAC and CNA) • Be sure internal staff have a chance to carefully review and discuss the report to ensure results and activities are followed. • Translate recommendations to action plans, including who is going to do what about the program and by when. • Include professional development to address MEP staff instructional duties and responsibilities • Create a specific plan to observe MEP funded services in an ongoing manner; making changes to plans based on observations (fidelity of implementation).

  27. Reporting Evaluation Results • Summarize the results in an executive summary; summary should include a an interpretation of migrant student performance on local and State formative and summative assessments (a list of test scores or grades is not helpful without this interpretation); • Be sure to record and store the evaluation plans and results in a location where it can referenced when a similar program evaluation is needed in the future.

  28. Pitfalls to Avoid in Evaluations • There is no "perfect" evaluation design. Don't worry about the plan being perfect. It's far more important to do something, than to wait until every last detail has been tested. • Don't report just the successes. You'll learn a great deal about the program by understanding its failures too! • Don't throw away evaluation results once a report has been generated. Results don't take up much room, and they can provide precious information later when trying to understand changes in the program.

  29. Questions?

  30. Georgia MEP on Twitter@GeorgiaMEP

  31. THANK YOU!Contact Information John WightProgram Manager Georgia Department of EducationMigrant Education Programjwight@doe.k12.ga.us404-463-1857 Omar Lopez-Nunez State Research & Development CoordinatorGeorgia Department of EducationMigrant Education Programolopez@doe.k12.ga.us404-463-1775

More Related