Evaluation Working Group (EWG)

1. Evaluation Working Group(EWG) Update November 2009

2. EWG History & Operations • Started in 2007, less active in 2008 • Reactivated in 2009 with hiring of NNE • Teleconference every week (or so): [Monday at noon PDT (3p.m. EDT)] • Leads: Rosanne Fortner & Chris Parsons • Members: 3 COSEE PIs, 6 COSEE Evaluators, 1 COSEE Advisor, CCO, NNE & NSF

3. EWG Mission • To serve the COSEE Network, its sponsors, Center Evaluators (CE) and National Network Evaluator (NNE) by providing information & advice on: - collective evaluation needs - existing resources and collaborative opportunities that document the extent to which COSEE goals and objectives are achieved, as well as the Network’s impacts

4. EWG Tasks • A forum for identifying collaborative opportunities, which includes: • Assist the NNE with identification of and mechanisms for integrating specific evaluation research being conducted in the Centers; • Establish the scope of CE input to the portfolio of COSEE evaluation products, research and lessons learned; • Establish with NNE the standards for reporting existing or new “returns on investment” data being collected by the CEs; • Assist in brokering relationships among CEs and between the NNE and CEs; and • Provide logistical support for projects involving multiple CEs and NNE. (2) Periodic review and assessment of NNE plans

5. “Bins” Document History: A Scheme for Cross-Center Data Gathering Related to Task: Establish with NNE the standardsfor reporting existing or new “returns on investment” data being collected by the CEs. “Bins” Document: Scheme for Cross-Center Data Gathering - Part I: Definitions and Demographic Data (Scientist/Educator Engagement) - Part II: Bins for Activities & Participants Written, reviewed, discussed, reorganized, revised and some piloting About a year in the making; consideration of Returns on Investment and Business Plan; at least 8 drafts

6. “Bins” Document – Why? • As a network, we need a standardized way of collecting common data across Centers • We need this data to be able to: • Answer questions about what we’re doing (internal and external questions) as a network • Provide data (part of our evidence package) for the Decadal Review See page 34 of Briefing Booklet

7. Information gathered across Centers should… • be easy enough to gather • result in information and findings that are compelling • be credible • be cost effective

8. EWG Request of Council • We’re asking you to • Approve the Bins document as COSEE’s framework for cross-Center data collection • Tell us if we missed anything – Are there any glaring errors or omissions in this document • Commit to cross-Center data collection • Allocate time/resources to cross-Center data collection efforts

9. EWG Next Steps re: Bins • Starting work on establishing protocols and implementation plan for cross-Center data gathering • Working with NNE to set up data base for collected data

10. Some questions framing the next phase of EWG “Bins” work • When do we start? What is the reporting cycle? What is the data-gathering time frame? • Who within COSEE will fill in the data? • How do we come up with an agreed upon electronic format? • How do we pilot this? • How do we train and who do we train to do this? • How will data be reported? Who will do the reporting? Who will have access to the reports? • How will or could this serve Center needs vs. Network needs over time? Will it be revisited over time? • How does the Bins data collection relate to Human Subjects/IRB? • How will we know if the data gathering meets the “test” on the first page of the document?