ECTAC Evaluators Group

1. ECTAC Evaluators Group August 7, 2019 Tampa, Florida

2. AM Agenda

3. Evidence Based Research

4. Evidence Based Research A Quick Review

5. ESSA defines the term “evidence-based” as… “an that demonstrates a effect on improving student outcomes or other relevant outcomes.” activity, strategy, or intervention statistically significant (Every Student Succeeds Act of 2015, 2015)

6. Evidence Cycle Monitor Implementation Source: U.S. Department of Education, 2016 Non-Regulatory Guidance: Using Evidence to Strengthen Education Investments

7. Evidence Based Tiers

8. AKA: Logic Model Based on high quality research findings or positive evaluation indicating that improved student outcomes are likely AND Has an evaluation plan / ongoing efforts to examine the effects

9. AKA: Correlational Well-designed, well-implemented study (statistically significant results) AND Determines a relationship but cannot make a causal claim

10. AKA: Quasi-experimental Well-designed, well-implemented study (statistically significant results) AND Has comparison groups (purposeful treatment and control)

11. AKA: Experimental Well-designed, well-implemented study (statistically significant results) AND Has comparison groups (purposeful treatment and control) AND Groups were randomly assigned (commonly referred to as Randomized Control Trials)

12. Activity • As a group, we will continue to add to the evidence base research/practices database • Select an article on a topic/area that interest you. • Using the decision tree – determine what Evidence Based Research tier to categorize the article. Note: Any determination made as a group is not permanent. It can be discussed, challenged, or changed. The idea is to start somewhere and build a resource for others.

13. Evidence Based Tier Selection Criteria Evidence Based Tier Selection Criteria Document

14. USED and FDOE Requirements

15. Area of Focus • Rationale • Measureable Outcome • Evidence-based Strategy • Rationale for Evidence-based Strategy • Action Steps (numbered)

16. Tier 4 to Tier 3 Strategy

17. Evidence Cycle Monitor Implementation Source: U.S. Department of Education, 2016 Non-Regulatory Guidance: Using Evidence to Strengthen Education Investments

18. Tier 4 to Tier 3 Strategy • Questions to consider moving forward: • Do you see your district conducting evaluations to move a selected strategy from Tier 4 to Tier 3? • Why or why not? • If there was an editable template, would it be used? • Why or why not? • Additional comments

19. Program Evaluation

20. CFR 34 200.26 Core elements of a schoolwide program OLD LANGUAGE: https://www.ecfr.gov/cgi-bin/text-idx?SID=c6fac7a6920ae8c32c7e6d9aa143e694&mc=true&node=se34.1.200_126&rgn=div8 (c) Evaluation. A school operating a schoolwide program must— (1) Annually evaluate the implementation of, and results achieved by, the schoolwide program, using data from the State's annual assessments and other indicators of academic achievement; (2) Determine whether the schoolwide program has been effective in increasing the achievement of students in meeting the State's academic standards, particularly for those students who had been furthest from achieving the standards; and (3) Revise the plan, as necessary, based on the results of the evaluation, to ensure continuous improvement of students in the schoolwide program. NEW LANGUAGE: https://www.ecfr.gov/cgi-bin/text-idx?SID=75a5f6d55e8099b5e4c18e4728e75c99&mc=true&node=se34.1.200_126&rgn=div8 (c) Evaluation. A school operating a schoolwide program must— (1) Regularly monitor the implementation of, and results achieved by, the schoolwide program, using data from the State's annual assessments and other indicators of academic achievement; (2) Determine whether the schoolwide program has been effective in increasing the achievement of students in meeting the challenging State academic standards, particularly for those students who had been furthest from achieving the standards; and (3) Revise the plan, as necessary, based on the results of the regular monitoring, to ensure continuous improvement of students in the schoolwide program.

21. Important Clarification on New Language • Continuous Improvement and On-going Monitoring  • Keep in mind that evaluation should occur at multiple points throughout the year, not once at the end of the year. For example, only using FSA data at the end of the year to evaluate an activity’s effectiveness does not suffice.

22. Continuous Improvement Model

23. Evidence Cycle Monitor Implementation Source: U.S. Department of Education, 2016 Non-Regulatory Guidance: Using Evidence to Strengthen Education Investments

24. Program Evaluation Discussion Topics

25. Program Evaluation Questions Are there commonalities/groupings of questions? • How was the program implemented? • What it implemented with fidelity? • What evidence was collected and reviewed to determine implementation effectiveness? • When, how often, and by whom was evidence collected? • If the program is not implemented with fidelity, what are the next steps? • What data will be collect to determine if the activity is achieving the desired outcomes? • How will progress be monitored throughout the year?* • When, how often, and by whom will the data be collected? • At the end of year, what are the recommendations? Continue as is, amend (and what will be amended), or discontinue? • Are there questions missing that should be considered? Many of these questions are called out in the Application

26. Program Evaluation Topics/Areas

27. Google Drive Working Lunch

28. PM Agenda

29. Exceeding Expectations • Discussion/Decision Topics: • Commonalities • Documentation • The process for collecting feedback during site visits • Timelines and date restrictions

30. Comprehensive Needs Assessment ESSA Section 1114(b)(6) The schoolwide program plan “is based on a comprehensive needs assessment of the entire school that takes into account information on the academic achievement of children in relation to the challenging State academic standards, particularly the needs of those children who are failing, or are at-risk of failing, to meet the challenging State academic standards and any other factors as determined by the local educational agency”

31. Comprehensive Needs Assessment: Code of Federal Regulations §200.26   Core elements of a schoolwide program (a) Comprehensive needs assessment. (1) A school operating a schoolwide program must conduct a comprehensive needs assessment of the entire school that— (i) Takes into account information on the academic achievement of all students in the school, including all subgroups of students under section 1111(c)(2) of the ESEA and migratory children as defined in section 1309(3) of the ESEA, relative to the challenging State academic standards under §200.1 and any other factors as determined by the LEA to— (A) Help the school understand the subjects and skills for which teaching and learning need to be improved; and (B) Identify the specific academic needs of students and subgroups of students who are failing, or are at risk of failing, to meet the challenging State academic standards; and (ii) Assesses the needs of the school relative to each of the components of the schoolwide program under section 1114(b)(7) of the ESEA. (2) The comprehensive needs assessment must be developed with the participation of individuals who will carry out the schoolwide program plan. (3) The school must document how it conducted the needs assessment, the results it obtained, and the conclusions it drew from those results.

32. Comprehensive Needs Assessment Discussion

33. Comprehensive Needs Assessment Sample • Comprehensive Needs Assessment District Sample for Discussion

34. Next Meeting • Dates • In person vs Virtual • Utilize Google Drive for agenda creation and topic leads • If you are participating in the Administrators’ PD day presentations, please be prepared for an hour and half virtual meeting next week.

35. Administrators’ PD Meeting