N-PASS Evaluation: Year 1

N-PASS Evaluation: Year 1 Goodman Research Group, Inc.

Overall Impressions • Positive attitudes about the project • Positive about the curriculum • Evidence of some impact on programming at participating science centers and 4-H agencies • Commitment to moving forward, looking forward to a smoother second year

Professional Development Model • Recruiting and maintaining relationships with 5 CBOs that met selection criteria • Conducting seven monthly workshops with CBOs • Making monthly site visits to the CBOs

Recruiting and Retaining CBOs • 4-7 CBOs/trainer recruited in Year 1 • 3 sites recruited 4 CBOs, and 3 sites recruited 7. • 80% of CBOs able to continue meeting project next year

Conducting Workshops • Trainers carried out 4-6 workshops in Year 1. • Some Trainers felt there were too many workshops required in Year 1. • On average, there were 8 individuals at any given N-PASS workshop in Year 1 (range 4-15). • On average, participating CBOs were well represented at the workshops. • For the most part, Trainers felt their workshops were similar to what was modeled for them at the national and regional trainings.

Number of Workshops “Feels like a production line.” “We’re doing more workshops than CBOs can handle.”

Making Site Visits to CBOs • Only one Trainer was able to make most of the CBO site visits expected by the project. • Two others completed more than half of their expected visits. • The remaining Trainers were able to make only a few of the visits expected of them.

Mentoring Model • Conducting regional trainings • Providing additional support • Making site visits to Trainers

Quality of Leadership & Resources • On a 5-point scale from poor to excellent, most trainers rated: • The leadership of the training network by EDC and LHS as very good • The mentorship they received as very good; and • The professional development resources developed by EDC and LHS as good

We had a hard time seeing which of the two goals for the project should be a priority in the beginning – professional development of CBOs or science activities for children. Our mentor is awesome – has a lot of compassion and a true understanding of what it takes to make true partnerships with the community a reality. The GEMS units were difficult for sites to deliver because they were not written for an after school audience.

Usefulness of Training & Mentorship • On a 5-point scale from not at all useful to extremely useful, most trainers rated: • The National Institute very or extremely useful in supporting their work with CBOs • Training at the mentor museum very useful • Monthly conference calls with mentors very useful • Site visits from mentors received mixed ratings

The National Institute was great for meeting everyone and forming an initial understanding of the project, as well as content for the first workshops. The training at mentor museum was great to address our individual problems. Site visits and monthly conference calls were supportive and nice but not crucial. If they [the mentor] did come and do a site visit, what would be their role?

N-PASS Outcomes CBO Implementation Science Center & 4-H Outreach Trainer Confidence Leading Workshops

CBO Project Implementation • Generally similar to what was modeled in workshops • On average, half carried out projects with consistent group of children at least once a week

Kids loved the activities and that was important for the CBO leaders to see. The facilitators are at VERY different places as far as their comfort and confidence leading the activities. Most of our program leaders had difficulty leading successful discussions and many left discussions out entirely during the first couple of projects.

Turnover in students and staff at some sites was painfully evident. The CBOs have a specific culture and setup and we need to work with them more to establish how they can introduce these kinds of activities in their setting.

Science Center & 4-H Outreach • 8 sites provided pre and post breakdowns of percentage time spent on different types of community outreach: • 5 of the 8 spent less time doing direct science programming with children and more time training CBOs to do science with children; another site spent more time doing both direct science programming and CBO science training and less time doing non-science outreach

Outreach cont.

Confidence Leading Workshops • After Year 1, Trainers felt more confident in 6 out of 6 areas related to leading workshops: • Introducing science activities to trainees • Demonstrating science activities to trainees • Modeling pedagogical techniques for trainees • Teaching science content to trainees • Mentoring trainees over time • Overall professional skills as a trainer

Increase in Confidence

Year 2 Evaluation • Pre and post surveys of workshop participants • Monthly surveys of workshop participants • Pre and post interviews with CBO directors • Site visits to regional trainings (interviews with trainers and mentors and focus group with trainers) • End of year surveys and interviews with trainers and mentors

N-PASS Evaluation: Year 1