1 / 17

Using Formative and Summative Evaluation to Improve Field Trip Programs

Using Formative and Summative Evaluation to Improve Field Trip Programs. Kimberly M. Burtnyk Amgen Center for Science Learning California Science Center. Think SCIENCE ! Pathways Components. Pre-/Post-visit classroom activities Curriculum compatible K8 CA standards

yaakov
Download Presentation

Using Formative and Summative Evaluation to Improve Field Trip Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Formative and Summative Evaluation to Improve Field Trip Programs Kimberly M. Burtnyk Amgen Center for Science Learning California Science Center

  2. ThinkSCIENCE! PathwaysComponents • Pre-/Post-visit classroom activities • Curriculum compatible • K8 • CA standards • Gallery worksheets (atypical) • “Chaperone Sheets” • Available free from our website

  3. ThinkSCIENCE! PathwaysGoals of the program • Help teachers implement curriculum-compatible field trips • Increase educational guidance for field trips • Increase interactions with staff • Help chaperones facilitate • Increase concept understanding through chaperone facilitation

  4. ThinkSCIENCE! PathwaysEvaluation Plan Formative EvaluationSpring 2002 Focus: Access, Use, Chaperone/student behavior Summative EvaluationTimeline Spring 2003 Focus: Cognitive gains

  5. Formative Evaluation Method • Recruiting • Groups with existing reservations • Questionnaires • Pathways usage issues • Field trip planning and behavior • Chaperone roles • Observations • Stopped/didn’t stop • Time spent--we want Pathways groups to stay longer • Panels Read, Discussions, Gestures

  6. Formative Evaluation Results Summary • Recruiting • 37 Schools recruited from March-June • Questionnaires • 82 Returned (37 Teacher 45 Chaperone) • Observations • 15 schools tagged for observation (8 PW, 7 Non-PW) • 72 Chaperones observed

  7. Formative Evaluation Questionnaire Highlights • Only 3 of 8 PW recruits used it • Reasons • Technical difficulties • Materials looked too complicated and long • Confused about where to find them • Unaware of the materials • Timing

  8. Formative Evaluation Observations Highlights Stopped/Did not Stop at Capsules • 72 Chaperones observed

  9. Formative Evaluation Observations Highlights Time spent at Capsules Mean t: 66 s Mean t: 138 s

  10. Formative Evaluation Observations Highlights Reading, Discussion, Gestures • Of the 35 chaperones who stopped...

  11. Formative EvaluationConclusions • Pathways usage was limited due to: • Access to computers/printers • Length and perceived complexity of the materials • Communication gaps between school administrators and teachers BUT... it did produce some significant behavioral changes in chaperones who used it

  12. How Did Formative Evaluation Impact the Program? • In-class/On-site activities separated • Shortened • Map removed • One activity removed (Gallery incomplete) • More direct link created • quick • easy • obvious

  13. Summative Evaluation Method • Recruiting • Talked directly to lead teacher • Materials sent to groups • In-class visit prior to field trip • Pre/Post Test • Questionnaires--dropped • Observations • Gestures dropped

  14. Summative Evaluation Results Summary • 298 Students Recruited • 217 had in-class visit • 254 Pre- and Post-tests completed • 28 PW • 226 Non-PW • Observations • 26 chaperones observed • ONLY 4 CHAPERONES USED PATHWAYS

  15. Summative Evaluation Conclusions • Jury still out on cognitive impact of Pathways • The numbers are too small • Will resume attempts next year • Pre/Post test shows impact of In-class Visit • Micrometeoroid question Chi Square p<0.004 for those who had visit vs those who did not

  16. What did Formative and Summative Evaluation Teach us About Pathways? 1. Even with intensive intervention and support, teachers did not use PW • There were special circumstances in the A&S Gallery • And timing, curriculum, age groups were an issue too BUT: • Are such materials worth spending $$ developing? • What do teachers really want/need, if anything? We don’t yet know. 2. Whether or not PW improves learning outcomes as the lit. suggests is still unknown • Warrants further testing and refinement of good and USEFUL field trip practices.

  17. What did Formative and Summative Evaluation Teach us About Pathways? 3. We really need to learn more about teachers and chaperones • Can’t assume that teachers have access to net sources • Chaperones might be willing facilitators 4. We can apply statistics to tell us about the use and effectiveness of the program • Further refining statistical techniques • Learning curve • Exciting prospects

More Related