Issues Surrounding the Evaluation of Teacher Internship Programs Donna Barrett

1. Issues Surrounding the Evaluation of Teacher Internship Programs Donna Barrett Center for Education Integrating Science, Mathematics & Computing Georgia Institute of Technology Atlanta, Georgia

2. …. a collaborative effort of corporations, universities and school districts designed to enhance mathematics and science experiences of Georgia teachers and their students. Since 1991, teachers have been placed in over 1100 GIFT internships throughout the state.

3. Goals of • Increasing the teacher’s personal knowledge base of science, mathematics and technology • Enhancing the teacher’s enthusiasm for teaching • Providing access to new techniques and procedures • Encouraging partnerships with business and education

4. The GIFT Experience - Commonalities • Common application • Internship in STEM-based work setting • Emphasis on “Real World” problem solving and reasoning • Development of Inquiry Based Unit – Action Plan

5. The GIFT Experience - Variables • Location of placement • University • Corporate • Informal science/public education • Goal of placement • ”Traditional” • Student research • International • Curriculum development • Length of placement: • 4-8 weeks

6. The GIFT Experience - Variables • Geography • Placements across Georgia • Teaching Level • High school vs. middle school • Teaching Field • Science, math, technology, social science • Type of school • affluent vs. low income, stable vs. changing, minority vs. majority

7. GIFT Evaluation Plan • An Advisory Board composed of 25 members from corporations, universities, and school districts • Utilizes the strategy of subcommittee phone conferences to provide guidance on program issues such as evaluation • Evaluation subcommittee has provided input on program evaluation • Worked with an internal evaluator to develop a GIFT Logic Model and develop assessments

8. Assessing Impact on Teachers • Participant Reactions • Numbers • Are participants satisfied? • Teacher and Mentor Surveys • Participant Learning • Action Plan Analysis • What have teachers learned about inquiry based learning? GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

9. Assessing Impact on Teachers • Participant Actions • How have they applied inquiry based learning in their classroom? • Surveys – Pre-Summer, Post-Summer & School Year Follow-up (Post-GIFT) • Alumni Survey • Classroom Observation GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

10. Using inquiry and hand-on activities Pre-GIFT N=47 Post-GIFT N=24 Pre and Post Surveys Based on surveys from: www.retnetwork.org

11. Assessing Impact on Teachers • Organizational Change • Change in school infrastructure • Teachers as agents of change • Alumni Survey • Focus Group Discussions • Classroom Observation GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

12. GIFT Alumni Survey • Searching for 668 former participants • GIFT database/Districts/Websites/Searches • 123 Fellows – No contact information • 545 Contact Attempts: 382 E-mail & 163 mailed; 36 E-mail and/or mail returned • 509 Total Sample Size • 127 - Surveys returned • Return Rate: 25%

13. To what extent, if any, do you feel that you experienced each of the following types of learning as a result of your participation in GIFT?

14. Leadership Roles • 16% reported they became a department or grade level chair person AFTER their GIFT experience • 21% reported they became a school district leader (Principal, AP or Coordinator) AFTER their GIFT experience

17. Limitations of Data • The data is self reported, although anonymous • Return rate of Alumni Survey – 25% • Sample sizes often vary on the pre and post surveys • Focus Groups have provided more specific information

18. Challenges in Correlating Student Achievement with Teacher Professional Development • Designing a scientifically valid study with appropriate controls • Appropriate measures of student achievement across grade levels, subjects • Access to student achievement data • Teachers changing schools, subjects and grade levels

19. Assessing the Impact on Students • Student Achievement • Connecting the dots through a literature review • Encouraging the use Research based best practices (Inquiry, Problem Based Learning, etc.) • Impacting teacher content knowledge GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

20. Future Evaluation Goals • Collect student achievement data of GIFT participants – proposed study with a local school district to collect preliminary data • Criterion-Referenced Competency Tests (CRCT) – Middle Grades • End of Course Test (EOCT )– High School • Biology, Physical Science, Algebra, Geometry • Aggregate data from specific class and compare to state average • Pre GIFT data, Post GIFT data, YR 1, YR 2 Post

21. Evaluation Tools/Resources • Columbia Program: http://scienceteacherprogram.org/ • Industry Initiatives for Science and Math Education (IISME): www.iisme.org • RET Network Surveys: www.retnetwork.org • SWEPT study: www.sweptstudy.org • TRE Conference on Research Experiences. http://omp.gso.uri.edu/CTRE/ • Visiting a High School Inquiry Classroom: How to Prepare and Observe: http://cse.edc.org/pdfs/products/observerguide.pdf

22. For more information: • Donna Barrett Program Director donna.barrett@ceismc.gatech.edu (404) 894.7530 www.ceismc.gatech.edu/gift