Improving Teacher Quality Grants: Needs Data and Evidence for Effective Partnerships

1. TitleIIA(3) Improving Teacher Quality Competitive Grants Program 2010-2012

2. Needs Data • Evidence Partners Meet Federal Criteria • Student Data • Indication of student needs based on available test scores or other pertinent student data

3. Needs Data • Participant Needs Survey Data • Standard Survey Form with place to add project-specific questions • Available for four core subject areas • Available in paper/pencil or electronic form • SAMPI compiles and returns data for use in proposal development (electronic survey only) • Submit summary of survey data and any additional data (student test scores, etc.) in narrative portion of proposal • Submit raw data with code numbers in database (data bases available for basic needs survey)

4. Needs Data • Other Data • Provide data pertinent to partner needs if available • Previous Research/Evaluation Findings • Describe previous actions taken by IHE partner to address identified needs • Describe value of proposed intervention based on previous findings • Category 1: Describe results of previous TitleIIA(3) project on which proposal is based

5. Needs Data • Teacher/School Commitment Forms • Informal “Understanding of Agreement” for teachers • Informal “Understanding of Agreement” showing partner school principal support • Participation in Technical Assistant Session—Part II • Prior to proposal submission to focus on needs data from survey and other

6. Evaluation

7. Evaluation Goals: • Improve programming through use of evaluation data and • Determine the impact of project and statewide effort on new and returning participants and identify strengths and limitations of the projects.

8. Project-Level Evaluation • Use common cross-site pre/post surveys, pre/post lesson observations, interview sample of participants, collect teacher artifacts, gather evidence of impact on students • Use additional instruments/procedures specific to project as desired to determine impact or gather implementation data • Gather and report participation/program data • Support a person with adequate time dedicated to evaluation • External or internal, with appropriate expertise and without major program coordination duties

9. CROSS-SITE STATE-LEVEL EVALUATION • Coordinated by SAMPI—Western Michigan University • Advise on local project use of pre/post surveys, pre/post lesson observation, interviews, teacher artifact collection, student impact data collection procedures • Advise on evaluation as requested • Coordinate cross-site meetings (see below) • Conduct observation of project PD activities • Compile data from across projects • Prepare periodic statewide reports

10. Elements of the Evaluation

11. Determine Impact on Teachers/Other Participants • Pre/post surveys (plus comparison with previous survey data for Category 1 projects) • Pre/post lesson observations (plus comparison with previous observation for Category 1 projects) • Sample of interviews of participants • Collect sample of teacher artifacts

12. DETERMINE IMPACT ON STUDENTS OF PARTICIPATING TEACHERS • Student test data as appropriate • Procedures to identify changes in student learning (pre/post tests, pre/post surveys, focus group interviews, assessment of student work) • Collect sample of teacher artifacts to show changes

13. PROJECT DIRECTOR AND PROJECT EVALUATOR PARTICIPATION IN REQUIRED CROSS-SITE ACTIVITIES • 1st six months—one face-to-face meeting in Lansing, one webinar • Remainder of project funding period—two face-to-face meetings in Lansing, two webinars

14. Required Common Reporting • Required Common Reporting • Participation levels and demographics, PD types, overall assessment of progress toward goals • Final evaluation report • End of project participant-specific data consistent with cross-site statewide data collection • Sharing findings at cross-site sessions

15. End-of-Project Reporting • Director Report • Use standard report format available in electronic version • SAMPI can provide Access database to projects to maintain participation records and to facilitate report preparation

16. Part #1 Sample Reporting Tables

17. Part #2: Director Perceptions of Project Accomplishments • Rate progress toward outcomes • Provide evaluation/other evidence to support rating

18. Part #2: Director Perceptions of Project Accomplishments-Sample Form Complete the following tables, one for each of your intended outcomes as per your proposal. Outcomes are statements of intended impacts or results that will occur as a result of your professional development programming or other project interventions.

19. Part #3: Nature of Professional Development/Interventions • Hours, participants, schedule by type of intervention • Role of college faculty/content experts • Problems planning, implementing project • Problems recruiting teachers

20. Part #3 Nature of Professional Development/InterventionsSample Form

21. Nature of PD/Interventions, Part #3, continued - Sample Form participation in your project. 3. Use the chart below to describe higher education faculty (both content and education faculty) or other external expert.

22. Part #4: Required Components of Evaluation Report • Data collection • Progress toward project outcomes • Lesson observation data • Teacher and student artifacts • Impacts on participating students • Effectiveness of project partnership • Other

23. Part #4: Required Components of a Project Evaluation Report • Prepared by your internal evaluator • Based on their evaluation work • Evaluation reports will vary • Should be appropriately labeled as Core Evaluation Report Components 1 - 6.

24. Part #4: Required Components of a Project Evaluation Report, continued Core Evaluation Report Component 1: Data Collection • Describe the data collection activities that occurred over the course of the project.

25. Part #4: Required Components of a Project Evaluation Report, continued Core Evaluation Report Component 2: Progress Towards Project Outcomes • For each proposed outcome of the project, briefly summarize progress made toward its accomplishment based on evaluation findings.

26. Part #4: Required Components of a Project Evaluation Report, continued Core Evaluation Report Component 3: Lesson Observation Data • Detailed findings from lesson observations should be included in the evaluator report

27. Part #4: Required Components of a Project Evaluation Report, continued Core Evaluation Report Component 4: Lesson Observation Data • Detailed findings from lesson observations should be included in the evaluator report

28. Part #4: Required Components of a Project Evaluation Report, continued Core Evaluation Report Component 5: Impacts on Students of Participating Teachers • For those projects that have gathered data related to impact on students, detailed findings should be presented in the evaluator’s report, if not part of the Progress Towards Project Outcomes (#2) above.

29. Part #4: Required Components of a Project Evaluation Report, continued Core Evaluation Report Component 6: Effectiveness of Project Partnership • Briefly describe the effectiveness of the partnership in implementing project activities. Provide evidence for your findings.

30. Other Reporting (Optional) • Report as appendix to core required report • Can include information Director wants to share about intervention, materials used, etc. • Can include additional evaluation or other pertinent data about project • Can include samples of teacher or student materials

31. Questions? • Contact information: • Kristin Everett – SAMPI, Western Michigan Unversity • kristin.everett@wmich.edu, (269) 387-3791