Enhancing Leadership Program: Website Evaluation for Geoscience Faculty

1. Evaluation andWebsite Overview Ellen Iverson Cathy Manduca

2. Agenda Goal: Help you capitalize on CE website and evaluation while making sure we collectively meet program requirements Part 1: Overview • Introductions • Evaluation - Our Framework • Taking Advantage of the Website

3. Evaluation • Formative: Improve our leadership program Improve Cutting Edge workshops • Summative Understand program (workshop and website) impact on geoscience faculty, community and culture Contribute to research base on effective faculty development

4. Logic Model

5. Small Sample 45 workshop participants (05) 38 community members (07) Participant Sample Exit workshop surveys Participant survey (05, 08) Pop-up website survey (05,06,08) Full Community Baseline survey (04, 09) Tiered Evaluation Model

6. What We Know: Workshop Experience • High satisfaction (9.0 out of 10) • Valued Experts at workshop Networking with other participants New teaching methods • Timely impact to teaching practice 50% use within 6 months 33% use within 3 to 6 months

7. What we Know:Website Impact • Supports: Changes in teaching Dissemination of best practices • Broader than workshop participants “are you familiar with the Cutting Edge site and how . . . you use these sort of things?” RESPONDENT: “Yeah. I’m telling you, a lot of people use them. We love them.” “Your web site for the course design tutorial looks great!  I'm going to work through it for my new course next semester.  I'd like to send the link to the people who participated in my workshop, too.” • Supports first time teaching course “In particular this spring, I had to teach a structural geology course, which is a course that I had, one, never taken, and, two, I’m a geophysicist, not a structural geologist, so it was a completely new type of topic for me to teach. And so I found there’s a whole section on structural geology on the Cutting Edge website that was just excellent. All of their examples, and movies of things, and being able to actually link to other professors’ websites where they had their exams and things like that on their website, so it was really, really helpful.”

8. Instruments • Roadchecks http://serc.carleton.edu/NAGTWorkshops/metacognition/workshop08/roadcheck.html • End of workshop surveys http://serc.carleton.edu/NAGTWorkshops/metacognition/workshop08/workshop_evaluation.html • Embedded assessments • Role of application • Activity review • Reflective pieces • Action plans • Participant follow up

9. Taking advantage of the CE website Two Major Roles for Website • Supporting workshop • Bringing resources and ideas from the workshop to the larger community Every workshop participant is required to submit something to the website

10. Examples • Supporting Preparation • Introductory Materials 15508 ,9372 ,9102Essays 28047,17558 • Activities28668 and Courses 28758 , Review28986 • Disseminating Results • Activities and Courses23349 • Session Products15509 • Synthesis Documents

11. Key Ideas in Design • Make use of existing resources early and often, in prep and in person- this builds awareness of website and motivates use • Consider what people can submit in advance that will simulatenously support workshop and build collection-it can’t be perceived as a waste of time or unrelated to the workshop • Think about session outputs - what is an endproduct that is reasonable, useful, and supports completing the discussion-if done well this focuses work in addition to creating a useable output for the website 28707