Evaluating Your Schoolwide Program

1. Evaluating Your Schoolwide Program 2011 Title Programs Conference June 14-17, 2011 Atlanta, Georgia

2. Presenters Ms. Elaine Dawsey Georgia Department of Education Title I Program Specialist edawsey@doe.k12.ga.us (478) 971-0114 Dr. Judy Alger Georgia Department of Education Title I Education Program Specialist jualger@doe.k12.ga.us (229)838-6067

3. Introduction, Learning Objectives, Annual Review Basics • Annual Review Basics • Measuring Your Progress • Annual Review Steps & Planning Ahead (Six-step annual review process)

4. Schoolwide Program Focus

5. Target Audience Includes: • School principals, assistant principals • Counselors • Instructional coaches • General education teachers • Special education teachers

6. Why do an annual review? • Federal requirement • More than just paperwork • Can guide your decisions • Part of a continuous planning process

7. Program Evaluation Requirements • Federal legislation (CFR 200.26(c)) requires that a school operating a Title I program to: • Annually evaluate the implementation of the Title I program, using data from state’s annual assessments and other indicators of academic achievement. • Determine the effectiveness in increasing the achievement of students, especially the targeted students. • Revise the plan, as necessary, based on the results, to ensure the improvement continues.

8. What should you include? • Guiding questions that match goals • High-priority issues • Relevant data • Jargon-free results and analysis

9. More Annual Review Benefits • Evaluates program implementation • Analyzes program effectiveness • Keeps focus on results • Helps you work toward AYP goals • Provides accountability

10. Purpose of Evaluation • The purpose of program evaluation is: • To determine if the identified strategies are contributing to the desired outcomes either in terms of improvement in student achievement, or increases in other activities that lead to increased student achievement such as greater parental involvement or more high-quality professional development. (ED guidance) • To determine if the effectiveness is increasing the achievement of students meeting the State’s academic standards. • To determine if the right students are served. • To determine if the right services are provided. • To verify, increase or change the impact of services for students.

11. Purpose of Evaluation • The purpose of program evaluation is: (continued) • To improve delivery strategies to be more efficient and effective. • To identify program strengths and weaknesses. • To assist district and school level leaders in making informed decisions. Especially important due to ARRA funds ending in 2011. • To address stakeholder questions and help them understand how effectively schools are meeting goals. • To increase understanding of improvement strategies.

12. Two Types of Evaluation • Ongoing Evaluation (quarterly/monthly): • Review data (weekly assessments, analysis of student work, teacher perception). • Discuss and analyze data • Instructional Strategies • Other factors • Summarize Progress Towards Goals • Progress Made • No Progress Made • Address any issues

13. Two Types of Evaluation • Annual Evaluation • Determine Progress Made For each Goal • Is there significant progress made? Why or why not? • Any trends? • Which goals, strategies are working? Which are not? • Review Needs and Goals • Have the needs and goals changed? • Any new needs that should be addressed? • Review components of SW or TA Plans • Are the components being implemented as indicated in the plan? • What updates/additions are needed? • Address any issues that arise • New concerns? • Will new issues need to be added to the plans?

14. Measuring Your Progress Gathering the right types of data

15. What are we evaluating • Progress toward schoolwide program goals • Efforts to address required program components

16. Monthly Data Meeting Benefits • Best practice • Helps you gather, analyze current data • Leads to strategic decisions • Keeps focus on student results

17. Monthly Data Meeting Tips Tip #1—Build a Team • Diverse leadership group • Include teachers, parents • Assign roles Tip #2—Celebrate Success • Schoolwide/administration • Within grade levels • Classroom • Individual • Report to stakeholders

18. Monthly Data Meeting Tips Tip #3—Follow an Agenda • Allows for pre-planning • Standardizes information • Provides focus • Documents efforts

19. Monthly Meeting Tip #3: Follow an Agenda • List the goal • Data review • Answer key questions • Summary • Timeline • New concerns

20. Monthly Meeting Tip #4: Pre-Planned Discussions • List overall data • Disaggregate data • Note changes in data • Write down questions

21. Monthly Meeting Tip #5: Use Timed Protocols • Staff members reports (1-2 minutes) • Team comments, asks questions (30 seconds each) • Reporter responds (1-2 minutes) • Take a vote • Create action step: assign roles (2-3 minutes)

22. Monthly Meeting Tip #6: Quality, Not Quantity Schedule strategically Limit outside interruptions Unplug technology Divide and conquer Split goals among meetings

23. QUIZ

24. True or False?Title I requires you to hold monthly data meetings and report your findings.

25. Answer: False

26. Annual Review Steps and Planning Ahead Who should do the review? • Internal or external reviewers • Outside reviewers preferred • Make it a collaborative decision

27. Step 1: Identify Purpose & Audience • Evaluate implementation and results • A broad audience: teachers, administrators, parents, students, community members

28. Step 2: Identify Issues and Develop Questions • Refer to monthly data meeting agendas • Evaluate all goals through multiple lenses • Review what matters • Remain realistic

29. Step 3: Identify Data Collection Tools • Quantitative data: Test scores, grades • Qualitative data: Survey responses, interviews, focus groups

30. Step 4: Collect Data • Communicate clearly • Explain the review’s • List the data being collected • Tell how the results will be used • Provide training • Seek large samples

31. Step 5: Analyze and Interpret Results • Describe school progress • Pinpoint individual strategies • Identify areas of need

32. Step 6: Create a Report • Be concise • Include background information, evaluation questions and procedures • Explain how data were analyzed • List findings and recommendations • Use clear language • Make it easily accessible

33. Next Steps • What should we continue? • What should we stop doing? • What needs to be done before the summer?

34. QUIZ

35. True or False?The annual review provides a broad picture of your schoolwide program, not details, such as individual strategies.

36. Answer: False

37. SWP Required 18 Components

38. Comprehensive Needs Assessment • Examine current data on the school’s demographics, test scores, teacher qualifications, attendance rate, discipline referrals, and survey results. • How have the school’s needs changed from the previous year as reflected by the data? • What needs have been met or unmet? • What contributed to meeting or not meeting the need? • What should continue, be revised or eliminated?

39. Scientifically Based Instructional Strategies • Closely examine the instructional strategies to ensure they are part of the overall school reform. • How have the reform strategies chosen strengthened the core academic program? • After reviewing/analyzing the data, did the strategies target the specific needs of the students? • Are the strategies addressing the quality and quantity of learning time? • Did the strategies and practices minimize removing students from the regular classroom during the regular school day? (Targeted Assistance Program) • Did program merit the money that is budgeted for the strategies/programs/activities?

40. Instruction by Highly Qualified Professional Staff • The school’s instructional staff constantly changes; therefore, a review of this section must provide for a plan to ensure that all teachers and paraprofessional are highly qualified. • Are all teachers highly qualified? If not, how will compliance be attained? • Are there procedures in place to ensure only highly qualified paraprofessionals are hired? • Are there plans in place to ensure that teachers and paraprofessionals remain highly qualified? • Are all paraprofessionals working under direct supervision of highly qualified teachers?

41. Professional Development • Professional development activities should align with the needs assessment to provide teachers and other staff with tools to ensure success for all students. • Are the professional development opportunities aligned to the identified needs? • Are the professional development activities provided directly impacting the identified needs of the school? How do you know?

42. Professional Development • Questions: (continued) • What evidence does the school have that the professional development opportunities are making a difference with achievement of students? • What embedded support and follow up is provided to ensure implementation and effective use of the learned skills and strategies by the staff? • Does the data indicate additional professional development is needed to improve teacher delivery? • What have the teachers indicated as their professional development needs?

43. Strategies to Attract Highly Qualified Teachers • Plans should include means of attracting and retaining highly qualified staff. • What is the teacher retention rate? • What is being done to ensure the most qualified teachers remain at the school? • Have you been able to fill vacancies with highly qualified staff? If not, should an incentive plan or different recruiting method be used? • What are the results of the staff survey in regard to school climate? • What marketing strategies are in place for recruitment?

44. Strategies to Increase Parental Involvement • Plans must include strategies to build the capacity of parents to assist in their child's education. • What methods are used to ensure that parents of targeted students are included in parental involvement activities? • Are there strategies in place to improve student learning for families? • How are the results of the annual survey used to evaluate the parental involvement strategies? How are the results shared? • What activities are offered to parents to build capacity? How does the school measure the effectiveness of these activities?

45. Assisting Preschool in a Successful Transition • A well thought out transition plan eases the stress of young children and their parents sets the tone of success for years to come. • Are the present preschool transitional plans working successfully? • Are all demographics and backgrounds of incoming students included? • What are the survey results in regard to pre-school transition? How is this data collected? • How is this data used to improve pre-school transition to elementary schools? • What transition plans are in place for students transitioning to middle school? To high school? Are plans successful? How do you know?

46. Measures to Include Teachers in Academic Assessment Decisions • Instructional planning by teachers must be based on the students’ academic progress toward the goals measured through appropriately aligned assessment. • Are school administrators/teachers able to interpret test data (state, benchmark and classroom assessments)? • Are teachers able to discuss their individual students’ needs and strengths in regard to state standards as measured by both the state and benchmark assessments? • How are teachers collaborating with other teachers, coaches, etc to help plan instruction and intervention based on the data from assessments?

47. Measures to Include Teachers in Academic Assessment Decisions • Questions: (continued) • How is the data being shared among staff? • What do the teachers say is working? Not working? • How is the progress of students in targeted assistance programs being monitored throughout the year? Are modifications being made in instruction based on this monitoring? • What criteria are used to exit or add students to the targeted assistance program?

48. Ensure Effective, Timely Additional Assistance • Students who experience difficulty mastering standards must be provided with effective, timely assistance. • How is the success of students receiving intervention measured? What kind of process monitoring is being conducted? • Are modifications being made in instruction based on the monitoring?

49. Ensure Effective, Timely Additional Assistance • Questions:(continued) • How are the students who need additional time and/or additional support receiving it? Is the additional time/support working? • If extended day or year is provided, are the students who are in most need attending? If not, what can be done to help increase participation or what can be done within the school day to help these students?

50. Coordination and Integration of Federal and Local Services and Programs • Procedures should be in place for the coordinating and integrating Title I resources with other resources to enable children served to meet the State standards. • Are all the programs and services available coordinated and integrated? • What strategies and/or activities are coordinated with other school improvement efforts? • Are the services and programs meeting the needs?