250 likes | 420 Views
Basic Skills, Outcomes & Fostering a Culture of Evidence and Inquiry. Rob Johnstone, Skyline College; Janet Fulks, Bakersfield College; Bob Pacheco, Barstow College; Ian Walton, Mission College. Sponsored by the William & Flora Hewlett Foundation Research conducted by the RP Group. Agenda.
E N D
Basic Skills, Outcomes & Fostering a Culture of Evidence and Inquiry Rob Johnstone, Skyline College; Janet Fulks, Bakersfield College; Bob Pacheco, Barstow College; Ian Walton, Mission College Sponsored by the William & Flora Hewlett Foundation Research conducted by the RP Group
Agenda • Opening Dialog: Why are we struggling? • Challenges identified through the Basic Skills Outcomes Capacity (BSOC) Study • Strategies for responding to these challenges • Closing Dialog: What would help colleges strengthen cultures of inquiry? Basic Skills, Outcomes, and Fostering a Culture of Evidence and Inquiry | 2009 | RP Group
BSOC FindingThere is a perception gap between administrators and faculty regarding the availability and use of assessment data. College administrators believe that information is more widely available and integrated into implementation planning than those who work more closely with students.
The Information Gap No data needs being met All data needs being met Information Gap Information Gap Gap
Opening Dialog Findings from the BSOC survey suggest we are struggling to get the right information to the right people at the right time. What are your thoughts? Basic Skills, Outcomes, and Fostering a Culture of Evidence and Inquiry | 2009 | RP Group
BSOC Finding Colleges are data rich and information poor.
The Data We Collect Data Not Being Used Data Frequently Being Used Unmet Unmet Data Needs Unmet Data Needs
The Action Gap “Our college is quite successful at gathering data and putting reports together based on that data. Where we struggle is with executing action plans as a result of this data. I don’t really think we do a good job of ‘doing’ anything with the data, or implementing programmatic changes or initiatives based on the data.”
BSOC Finding There is a pattern of gaps in the use of data and evidence across community colleges – gaps that might be inhibiting institutional learning.
Institutional Data Fills Two Important Gaps (Poor Planning) (Good Planning) (Good Performance) Improved Processes & Performance Standard Level of Organizational Awareness Elevated Level of Organizational Awareness Knowledge Gap Performance Gap Strategic Planning Function Institutional Effectiveness & Student Success Function Institutional Outcomes and Benchmarks Classroom and Service Area Assessment
BSOC Finding Colleges may be focusing on the wrong data.
Typical Twenty Year Trend for Common Institutional Outcome Measures Retention Rate Success Rate • Consider the multitude of changes over this period: • - faculty/staff turnover • program successes/failures • changing student demographics • budget contractions/expansion • evolving state & accreditation mandates • leadership turnover • and yet these performance metrics remained amazingly stable. What does that tell us about the usefulness of these metrics in setting institutional strategies?
Assessment and the Organizational Learning Map Learning Pathways under Three Assessment Frameworks Entry Point (Three levels below transfer) • Assessment within the Classroom • Assess both during and at the end of the semester • Modifications are both structural & procedural • Change is frequent & free form Within Math 100, did a group assignment produce stronger learning than a lecture? Did the learning community for Math 100 work? Desired Outcome (Pass transfer-level course) • Program Level Assessment • Assess at the end of the semester • Modifications tend to be structural • Change is slow & deliberate
BSOC Finding Institutional research supports institutional metrics to a higher degree than classroom assessment.
Level of Involvement in the Use of Data High Level of Involvement Low Level or No Involvement Chief Instructional Officer Classroom Faculty
BSOC Finding Much of what is needed to support good decision making lies beyond the current grasp of our institutional databases.
BSOC Finding Over the last decade, most of the progress in gathering and utilizing data has been at the institutional level.
Degree to Which Evidence is Used in Decision Making Closing the Loop Robust Planning & High Levels of Innovation Strong Sense of Direction but Poor Understanding of Program Performance Outward Looking Forward Looking High Cluster of California Community Colleges Poor Planning & Weak Understanding of Programs Much Context-Specific Knowledge but Poor Integration Inward Looking Use of Data for Institutional Metrics Low Low High Use of Data in Classroom/Service Area Assessment
BSOC Recommendations 1 Research should be brought to the intervention level, at the classroom.
Expanded Definition of Research Capacity Production Closing & Widening the Loop Dissemination Translation Interpretation
BSOC Recommendations 2 • Research offices should receive support that will enable them to address multiple domains of research capacity. • Technical assistance should be provided to colleges that will help them to strengthen cultures of evidence and inquiry. The Bridging Research, Information, and Culture (BRIC) Project
The BRIC Project Developing a Better Understanding of How to Close the Performance Gap (Poor Planning) (Good Planning) (Good Performance) Improved Processes & Performance Standard Level of Organizational Awareness Elevated Level of Organizational Awareness Knowledge Gap Performance Gap Strategic Planning Function Institutional Effectiveness & Student Success Function Institutional Outcomes and Benchmarks Classroom and Service Area Assessment
Closing Dialog • The BRIC project will allow us to provide tailored support at colleges. • How could an external team of faculty, administrators, and researchers help deepen the culture of inquiry at your institution? Basic Skills, Outcomes, and Fostering a Culture of Evidence and Inquiry | 2009 | RP Group
Contact & Information RP Group Website www.rpgroup.org BSOC/BRIC Project Contacts Robert Johnstone, Project Director johnstoner@smccd.edu Priyadarshini Chaplot, Project Coordinator pchaplot@rpgroup.org Basic Skills, Outcomes, and Fostering a Culture of Evidence and Inquiry | 2009 | RP Group