Kurt Ewen Data Summit - Bakersfield College March 14, 2014

1. From Evidence to Actionable Information: Institutional Practices in Support of Student Success Kurt Ewen Data Summit - Bakersfield College March 14, 2014

2. From Gathering to Using Assessment Results: Lessons from the Wabash National Study National Institute for Learning Outcomes Assessment (NILOA)

3. From Gathering to Using Assessment Results • “Most institutions have routinized data collection, but they have little experience in reviewing and making sense of data. It is far easier to sign up for a survey offered by an outside entity or to have an associate dean interview exiting students than to orchestrate a series of complex conversations with different groups on campus about what the findings from these data mean and what actions might follow.”

4. Language in an Age of Accountability • The language we use makes a difference • Data driven / supporteddecision making • Culture of evidence / inquiry • Evidence based (informed) improvement

5. Google Image search • Data Driven Decision Making, Higher Education • Culture of Evidence

6. Institutional Practices:The promise of Habits & Rituals • Practice makes permanent (not perfect) We are what we repeatedly do. Excellence, then, is not an act, but a habit. Aristotle

7. Institutional Effectiveness &Regional Accreditation – Institutionalized Habits Identify Intended Outcomes Changes based on reflection Continuous cycle of Improvement Assess for Progress The process of reflection is rarely given sufficient attention Reflect on Results Office of Institutional Assessment - September 2007

8. SACACOC &Institutional Effectiveness Example - Comprehensive Standard S 3.3.1 • The (1)institution identifies expected outcomes, (2)assess the extent to which it achieves these outcomes, and (3)provides evidence of improvement based upon analysis of the resultsin each of the following areas: • 3.3.1.1 – educational programs, to include student learning outcomes • 3.3.1.2 – Administrative Support Services • 3.3.1.3 – Education Support Services

9. Achieving the DreamA National Strategy with Local Strategies Developing institutional habits on the way to Culture

10. AtD @ValenciaGetting Started 3 College Wide Strategies + Campus Strategies 89 Good Ideas Ripeness Effectiveness Scalability

11. Climateof Innovation Level I Level II “Eye for Evidence”: More rigorous at each level. 1000’s of opportunities tried. Maintain a Research and Development Component. 100 are selected for support as Phase I Innovations. “Angel Capital Stage” Prototype 10 supported as Phase II Innovations. “Venture Capital Stage” Pilot Implementation (Limited Scale) Level III 1 or 2 are brought up to scale and Institutionalized. Institutionalization Strategy Innovation Management System Hypothesis Valencia’s challenge has been moving from Level II to Level III. Level II Innovations must be scalable and must show potential to bring systemic change and “business-changing results.” Standard of evidence and reflection increases at each level.

12. Strategy Four Year Implementation Timeline 2005-06 2006-07 2007-08 2008-09 2009-10 Phase I Supplemental Learning MAT 0012, 0024, 1033 1600 FTICs ENC 1101, POS 2041, MAC 1105 Phase II Linked SLS Course (LinC) SLSLinC w/ MAT 0012, 0024, 1033 & w/ ENC 1101, POS 2041, MAC1105 as option 600 FTICs Phase III Linked Interdisciplinary Courses (LinC) ENC1101, POS2041,MAC1105 w/ Other Courses 300 FTICs Phase IV Required SLS Enrollment 3Prep Course Mandates 2Prep Course Mandates 2500 FTICs

13. Habits on the way to Culture • A “culture of evidence” requires the development of an institutional practice that (1) gives careful consideration to the question being asked (the articulated outcome) and (2) gives careful consideration to the data needed in order to answer the question.

14. Habits on the way to Culture • Structured reflection and dialogue allows for data about the results of student success oriented strategies to be transformed into meaningful information • The “meaning” of learning and student success oriented data is not generally self-evident • Meaningful information creates the possibility for action

15. Dialogue vs. DiscussionAn Etymological distinction • Dialogue: • Seeing the whole among the parts • Seeing the connections between parts • Inquiring into assumptions • Learning through inquiry and disclosure • Creating shared meaning • Discussion: • Breaking issues / problems into parts • Seeing distinctions between parts • Justifying / defending assumptions • Gaining agreement on one meaning / result

18. Dialogue & Six Common Questions • What connections/ patterns do you see in the data? • What else do you need to know? • Items of promise? • Items of concern? • Who needs to be brought into the conversation to help clarify the data? • Based what we “know,” what needs our attention (Ideas for possible action)

20. Habits on the way to Culture • Meaningful information creates the possibility for consensus about lessons learned and a shared vision / plan for the future • No data should be shared as information until it has been processed in a collaborative and thoughtful way. 

21. Habits on the way to Culture • A culture of evidence is one that seeks data supported decisions. • Data driven decision making runs the risk of over looking / underestimating the human factor which is very often concealed by the desire for statistical significance. • Data driven decision making runs the risk of removing us from the decision making process • A on word anecdotal evidence

22. Habits on the way to Culture • Meaningful information from data does not emerge from a single data point but from the intersection of multiple and varied (quantitative and qualitative) sources of data • For really significant questions about student success efforts there is no silver bullet • This is the kind of process that few organizations are able to manage

23. “Meaningful Information” found at this intersection often appears to be more of a hypothesis or experience informed hunch

24. Habits on the way to Culture • Data supported decisions concerning student success oriented programs requires a consideration of "meaningful improvement" and may require balancing all or most of the following: • Statistically significant improvement in target measures. • Reflection on the “human impact” • Economic efficiency in relationship to difficulty of the task at hand. • A consideration of perception as it relates to benefit versus cost.

25. Habits on the way to Culture • The sharing of Information should reflect standards of scholarly communication and evidence • Biases and conclusions about the information should be clearly articulated • Open and unanswered questions should be articulated (but should not be allowed to stop the process) • Differing perspectives on the meaning of the information should be given equal time • The use of programmatic and academic jargon should kept to a minimum • The visual presentation of information should be monitored for consistency

26. Habits on the way to Culture • The best insights often come as an unintended result of simple questions asked about things you were not planning to question.

27. Habits on the way to Culture • How are our new students doing? • Data was provide on FTIC Degree Seeking Students • Who are our new students? • Development of our Philosophy statement on the New Student • All Students with less than 15 College-level credits at Valencia • Who are our new Students?

28. Who are our New Students?

29. How are our New Students Doing?

30. Presenter Kurt Ewen AVP, Institutional Effectiveness and Planning Valencia College 407-582-3413 kewen@valenciacollege.edu