E N D
Evaluation Data Interface Jonathan Waterhouse & Meghan Bathgate Special thanks to Carl Haberfeld
Summer Institutes (SI) Evaluation • Our Evaluation Approach • Systems-based approach: teachers, departments, institutions • Program modeling: progression from personal teaching change to STEM reform • Translate findings: SI curriculum and publications • Data Collection and Storage • 300+ research surveys in Qualtrics • 10+ years of data • 2 million+ data records • 200+ papers • The SIs Intensive multi-day workshops for STEM educators focused on evidence-based teaching principles Since 2004, ~2,000 faculty from 400 institutes have attended
SI Evaluation Components Evaluation Questions Program Model Over 200 Papers Data
The Evaluation Data Interface (EDI) • The EDI is an web interface which connects all evaluation components in a single, interactive environment (i.e. Evaluation Questions, Program Model, Papers, and Research Data) • Different program stakeholders have different needs for viewing data and evaluation information • STEM Evaluation Team • Summer Institute Leaders • Survey Participants • Funders (Helmsley and HHMI) [DEMO]
Connecting Research Data to the EDI • We developed a custom piece of software “Q-Charter” which: • Reads raw data from Qualtrics • Manipulates and compares data from any survey automatically • Survey data comparison is not a feature Qualtrics offers, but was important to the efficiency of our evaluation efforts • Across multiple years, SI locations, and survey types
Q-Charter: Features and Capabilities • Capabilities: • Select how many and which Qualtrics surveys to compare • Remove questions for report or de-identification purposes • Redact personally identifying information from text entries • Automatically creates html reports which we then upload to the EDI
Activity & Discussion • 1. Gather into small groups of 3-4 • 2. On a notecard, answer the following: If you use Qualtrics, how can Q-Charter could help you or your team to visualize data? If not, how could you see data visualization supporting your work? 3. Questions