The Power and Potential of Analytics

1. The Power and Potential of Analytics Dr. Linda Baer Senior Program Officer Bill & Melinda Gates Foundation November 15, 2011

2. The Power of Analytics Optimizing student success is the “killer app” for analytics in higher education. Intelligent investments in optimizing student success garner wide support and have a strong, justifiable return on investment (ROI). Moreover, improving performance, productivity, and institutional effectiveness are the new gold standards for institutional leadership in the 21st century. Enhanced analytics are critical to both. Donald Norris, Building Organizational Capacity for Analytics, 11/11/2011

3. Facts • President’s challenge by 2020 to have the highest proportion of college graduates in the world • Nearly 2/3 of jobs in the future will require a college education. • Nation has fallen from first to tenth in the world in degree completion. • Business has had to provide training to entry-level workers in order to prepare them for the workforce.

4. The Great Paradox: Need for Breakthroughs in Performance 90% of community colleges in 2010 and 69% in 2011 Additional 300k to 1 million credentials needed per year Demands of globalized, information economy Deeper learning outcomes Higher enrollments More completions Rising expectations Declining family ability to pay Budget cuts Limited seat capacity Constrained resources 32% of community college students unable to enroll in classes; CA turning away up to 250k students per year; 675k total! 58% of community college budgets cut in 2011-2012; 41% of cuts >5%; long-term competition with healthcare Student load debt now greater than all consumer loan debt. 450% increase in tuition over 30 years Source: 2011 Community Colleges and the Economy, AACC/Campus Computing Project, April 2011; Community College Student Survey, Pearson Foundation/Harris Interactive, Field dates: September 27th through November 4th, 2011

5. Definitions of Analytics • Academic analytics combines large data sets, statistical techniques and predictive modeling to produce “actionable intelligence.” Campbell, DeBlois, and Oblinger. 2007. EDUCAUSE review July/August. • Action analytics produces actionable intelligence, service-oriented architectures, mash-ups of information/content and services, proven models of course/curriculum reinvention, and changes in faculty practice that improve performance and reduce costs. • Analyticsis about pursuing the improvement and optimization of institutional performance along all dimensions. Norris, Donald, Linda Baer, Joan Leonard, Lou Pugliese, Paul Lefrere, “Action Analytics: Measuring and Improving Performance That Matters,” EDUCAUSE Review, Jan/Feb 2008.

6. Definitions of Analytics • Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs • Analytics draws from, and is closely tied to, a series of other fields of study including business intelligence, web analytics, academic analytics, educational data mining, and action analytics. http://en.wikipedia.org/wiki/Learning_analytics

7. Disrupting and Transformative Analytics • Accountability Analytics – external policy makers and publics set standards for accountability and comparability, individual institutions judged according to performance • Performance Measurement and Improvement Analytics – • To meet accountability standards institutions must embed analytics into their processes and continuously strive to measure and improve performance.

9. QUESTIONApplying Analytics What student outcome data are you using now and in what ways?

10. The Power of Analytics

11. Donald Norris, Building Organizational Capacity for Analytics, 11/11/2011

12. The Analytical DELTA • D Accessible, high quality Data • E Enterprise Orientation • L Analytical Leadership • T Strategic Targets • A Analyst Analytics at Work. Davenport, Harris and Morison.2010

13. QuestionApplied Analytics What best characterizes your current use of analytics?

14. Complete College America By sharing responsibility for success, we can turn broken dreams and missed opportunities into college graduates and economic success. • Students must work hard, make good choices, and stick with it. • Colleges and universities must make graduation, not head counts, their measure of success. And they must align to the needs of today’s students. • States must knock down obstacles, across entire educational systems, that unnecessarily block paths to college completion – and they must encourage and hold accountable institutions and students for measurable progress. http://www.completecollege.org/alliance_of_states/

15. Metrics for Success • Outcome metrics • Progress metrics • Adult learner metrics • Noncredit metrics

16. Outcome Metrics • Degrees and certificates awarded • Graduation rates • Transfer rates • Time and credits to degree http://www.nga.org/Files/pdf/1007COMMONCOLLEGEMETRICS.PDF

17. Progress Metrics • Enrollment in remedial education • Success beyond remediation • Success in first-year college courses • Credit accumulation • Retention rates • Course completion http://www.nga.org/Files/pdf/1007COMMONCOLLEGEMETRICS.PDF

18. Focus on Adult Learner Progress in Degree Completion • Target strategy for adult learners • Develop segmented services for working adults • Monitor progress and interventions • Strengthen advisory business/industry committees • Build more contextual learning • Internships and Apprenticeships Counting the Hidden Assets, Business Roundtable, 2009. www.businessroundtable.org

19. Measuring Noncredit Activity • Taxonomy of noncredit offerings and outcomes • Outcomes and metrics • General basic academic skills • General workforce skills • Sponsor-specific basic skills • Sponsor-specific workforce development Counting the Hidden Assets, Business Roundtable, 2009. www.businessroundtable.org

20. QUESTIONApplying Analytics What questions about optimizing student success or increasing performance and productivity are most important to you?

21. The Potential of Analytics Analytics is the “universal decoder” for education reform.

23. Achieving the Dream • Achieving the Dream is a national nonprofit dedicated to helping more community college students succeed, particularly students of color and low-income students. • Committed leadership • Use of evidence to improve programs and services • Broad engagement • Systemic institutional improvement • Developmental education • First year experience • Curriculum and instruction • Retention and support services • Equity http://www.achievingthedream.org/CAMPUSSTRATEGIES/PROMISINGPRACTICES/default.tp

24. Washington State Student Achievement Initiative Metrics • Building towards college-level skills (basic skills gains, passing precollege writing or math) • First year retention (earning 15 then 30 college level credits) • Completing college-level math (passing math courses required for either technical or academic associate degrees) • Completions (degrees, certificates, apprenticeship training) Tipping Point, David Prince & Davis Jenkins http://ccrc.tc.columbia.edu/Publication.asp?uid=600

25. Supporting Student Success: Preventing Loss, Creating MomentumA System Designed for Student Completion ENTRY PROGRESS CONNECTION COMPLETION Interest to Application Enrollment to Completion of Gatekeeper Courses Enrollment to Completion of Gatekeeper Courses Complete Course of Study to Credential with Labor Market Value

26. The Analytics Capacity Gap • Gap between articulated institutional needs and vendors offerings • Analytics capacity compared with emerging expectations – Analytics IQ • Collaboration gap • Talent Gap Donald Norris, Building Organizational Capacity for Analytics, 11/11/2011

27. QuestionApplied Analytics If you could tell vendors about what tools or resources you would like to see that would best meet your analytics needs, what would you say?

28. What if Steve Jobs had been an educator?

29. Amazon features

30. Educational Positioning System

31. Educational Positioning System • Map-in starting point and destination • Routes to completion • ROI – ROV • Time to destination – progress • Fuel for the journey • Travel time to “norm” for the destination • Highway for optimizing student success Mike Mathews

32. “If educational completion is one of the most prized and valuable destinies for every American student, surely we need to innovate and market a vision that leverages the technologies and analytical tools that will completely eradicate the educational risks of taking wrong-turns, running out of academic gas, miscalculating the distance, under estimating the costs, and the inability to have a ‘norm’ to compare a personalized educational journey against.”  Michael Mathews