Noyce Program Participatory Evaluation

1. Noyce Program Participatory Evaluation FRANCES LAWRENZ JIM APPLETON MARJORIE BULLITT BEQUETTE ANN OOMS DEENA WASSENBERG University of Minnesota

3. What Existing Research Shows • Attention to recruitment and retention has varied over time • Different types of incentives to affect recruitment and retention have been tried • Alternative certification is one type of incentive and has a variety of definitions • Salary total amount and comparatively is a clear incentive for STEM teachers • Retention in a school is highly dependent on the context of the school (supportive colleagues and administration, optimism and focus on student success)

4. Theoretical Framework: Noyce Logic Model • The Noyce Program Ideal • Depicted by the main path as well as bold headings preceded by addition signs (e.g., “+Plan to teach”) • Decision points en route to becoming a STEM teacher • Indicated by diverging routes from the main path describing alternative options and the Noyce Ideal in bold headings • Dashed boxes denote retention/recruitment by school or program • Important STEM major decision factors • Influenced by attributes of the candidate, pre-service program, and school/district (depicted as bulleted lists on the main path) • Depicted as thought bubbles emerging from the decision point

5. Logic model

6. Evaluation Variable Data: Summary • Program itself:many are gathering data on recruitment, demographics, and student performance; fewer on program retention • Post-program monitoring:several to many are gathering data on teacher effectiveness, monitoring/fulfillment of scholarship requirements, school/district retention, teacher transition experiences/support, and inter-program/institution coordination; fewer on school/district recruitment and characteristics (including teaching assignment characteristics) • Ways of gathering data: mostly formative and summative program effectiveness, and self-report (caution)

7. Use of Noyce Monitoring Data • Whether the institution uses selected activities such as mentoring, early field experiences, etc. • Information on the students in the program • e.g., the numbers of students graduating, the numbers of career changers • Information on the applicants for the Noyce awards • e.g., numbers, GPA, support provided • Information on the faculty involved in the program • e.g., numbers, discipline areas and roles. • Information about both current and prior recipients of the Noyce funding. This includes • demographic information, GPA, prior background, intended teaching areas and levels, levels of funding, and completion dates. • prior recipients also includes if and where they are teaching and if so, what. • List of the districts involved with the Noyce program.

8. Ideas for an Evaluation of Noyce • Precise definitions of variables/separate district and program recruitment and retention • Numbers and demographics at each stage in the process of developing a teacher • Description of the experience at each stage • Assessment of the state of the candidate (performance, attitudes, plans, beliefs) at each stage • Examination of interactions • Longitudinal studies of persistence, activity and impact

9. Issues Raised at Conference Thank You! • Recruitment both program and district • Retention in district (program?) • Training-practice alignment • Sustainability • Definitional concerns e.g.,LEA, consistent • Type of evidence • Community building • Long term considerations

10. Existing Resources: Some copies at Tables • Evaluation instruments/processes presently used in Noyce • CETP instruments • LSC instruments • CCSSO instruments • Others??

11. Steps in Planning an Evaluation • Brainstorm evaluation questions • Focus through: audiences, existing research and data, importance, and effort • Determine information necessary to answer the questions • Establish indicators for the information including data sources and collection processes • Gather, analyze and share the information

12. Group Work—15 Minutes: recruitment, retention, programs, and partnerships • Brainstorm potential evaluation issues/questions about your topic • Prioritize the list • Speculate on information needs to answer the questions and possible indicators • Consider possible data sources, processes, and instruments • Prepare a 5 min summary for full group

13. Closure and Next Steps • We will combine and post the suggestions on our web site • We will structure continuing discussions with smaller groups of people on targeted issues via web or telephone (volunteers) • We will host another virtual conference in early fall to discuss group progress and next steps

14. Contact Us! • Frances Lawrenz • lawrenz@umn.edu • Marjorie Bullitt Bequette • marjb@UMN.EDU • R3 URL: • http://education.umn.edu/EdPsych/NOYCE/ • Additional input to the site is welcomed