Weaving Assessment and Accountability into the Institutional Fabric

1. Weaving Assessment and Accountability into the Institutional Fabric Victor M. H. Borden, Ph.D. Associate Vice President University Planning, Institutional Research, and Accountability (IU) Associate Professor of Psychology (IUPUI) Past President (AIR) Devoted Fan (M.O.U.S.E.) Former Spy (U.N.C.L.E) What if Everyone Actually Bought In? 2006 Arizona AIR Conference – March 31 – Phoenix

2. Overview • What the H-E- is he talking about? • Is there some kind of a problem? • Bought into what? • What is it we do again and why? • What might we be doing and how?

3. What’s Going On? • Is your workload picking up? • Is there less, if any down time? • Do you have time to clean up and decompress/reflect between projects? • Is there such a thing as “between projects” or just “among projects?”

4. The Challenge • Increased demands for what we do • External demands • Improved literacy • Increased expectations as to what can be done • Technological advancements • Expectations for self-service • Our own effectiveness • No notable increase in resources

5. What Exactly Do We Do? • What is IR? • Who uses IR? • For what?

6. The Many Faces of IR

7. Arizona Faces of IR

8. As We See Ourselves?

9. As Others See Us?

10. What is IR? …research conducted within an institution of higher education to provide information which supports institutional planning, policy formation and decision making. • Saupe, 1990 – The functions of IR

11. Brought Into What? • Using data, information, evidence, analysis to… • Plan – Figure out what to do • Implement – Figure out how to do it • Evaluate – Determine how well it is being done • Improve – Figure out how to do it better

12. Who Uses It? • Senior administrators • Administrative directors • Administrative staff • Campus committees • Deans and department chairs • Faculty senates/councils • Faculty • Students and student groups • State, federal and commercial agency staff

13. Who Could Use It? Every-friggin-body

14. Enrollment management Resource allocation Management Program improvement Process improvement Planning and budgeting Accountability Climate assessment Student learning outcomes assessment Environmental scanning Economic impact Faculty salary equity Space utililization Grant development Marketing Knowledge management Compliance reporting Data administration and warehousing Student engagement Alumni engagement and cultivation Used for What HE Functions?

15. What Can IR Be Used For? Every-friggin-thing

16. What is it we Really Do? • Sit at our desks, producing tables and charts to hand off to those who requested them? • Facilitate organizational learning

17. The IR Credo I realize that I will not succeed in answering all of your questions. Indeed, I will not answer any of them completely. The answers I provide will only serve to raise a whole new set of questions that lead to more problems, some of which you weren’t aware of in the first place. When my work is complete, you will be as confused as ever, but hopefully, you will be confused on a higher level and about more important things

18. A New Definition of IR • The primary function of institutional research is to facilitate organizational learning for the continuous improvement of higher education institutions and systems.

19. What’s important is not what the data say, It’s what we say and do about the data ~Victor Borden

20. A Learning Paradigm • Planning, evaluation, and improvement all fit within a rational model • Learning incorporates uncertainty, ambiguity, and multiple styles • Individual learning and organizational learning are compatible concepts • Learning is mission critical for all educational institutions

21. Norton & Kaplan’s BSC • Financial performance • Customer service and satisfaction • Process effectiveness and efficiency • Organizational learning

22. Senge’s 5 Dimensions of Organizational Learning • Personal mastery • Mental models • Building shared vision • Team learning • Systems thinking

23. Single- and Double-Loop Learning • Learning is the detection and correction of error (unintended consequences) • “Governing Variables” are those things what we feel are important to keep within acceptable limits • “Action Strategy” is what we do or plan to do to keep the governing variables within limits • “Consequences” are the intended and unintended outputs and outcomes • Intended: confirm our theory in use • Unintended: suggests error in our theory in use

24. Single-Loop Learning • Governing variables not called into question • Adjustments made to action strategies at best • Defense mechanisms can readily arise to maintain single-loop learning Governing Variables Action Strategies Conse- quences

25. Double-Loop Learning • Questioning the role of the framing and learning systems which underlie actual goals and strategies • Reflection is fundamental • Basic assumptions are confronted • Hypotheses publicly tested • Falsification is sought • Ego is laid aside Governing Variables Action Strategies Conse- quences

26. Model I and II Org Learning • Single- and double-loop learning at the organizational level • Model I: Organizational members prescribe to a common theory in use • Organizational policies and practices inhibit change • Model II: Governing values, policies, and practices promote double-loop learning

27. A Model I Learning Organization • Governing Variables • Tow the line • Win at all costs • Suppress negative feelings • Emphasize rationality • Action Strategies • Control environment and task unilaterally • Protect self and others unilaterally • Discourage inquiry • Consequences • Defensive relationships • Low freedom of choice • Reduced production of valid information • Little public testing of ideas

28. A Model II Learning Organization • Governing Variables • Valid information is most important • Free and informed choice • Shared internal commitment • Action Strategies • Shared control • Participation in design and implementation of action • Consequences • Minimally defensive relationships • High freedom of choice • Public testing of ideas

29. Domains of HEI Information Use • Two dimensions • Commonality of goals and objectives • Intentionality of information use • Four Quadrants • Model I • Model II • Disciplinary Scholarship • Nothing doing

30. Individual/Private Each person works toward their own purposes Everyone works toward the same purposes Organizational/Shared Commonality of Goals and Objectives

31. Intentionality of Evidence Use Subjective/biased Looks for and accepts evidence that supports a given conclusion Objective or balanced approach to identifying and examining evidence Objective/unbiased

32. HEI Information Use Domains Organiz’l/shared Intentionality of Information Use  Commonality of Goals/Objectives  Individual/private

33. Beyond Information Provider • It is one thing to give people the information they think they need to make decisions • It is quite another to engage our colleagues in using information to figure out how to improve our collective lot

34. How Might we Expand Our Impact • Borrowing from colleague’s models • Structural arrangements • Brokering

35. Colleague’s Models • Library • Collections; reference; information literacy; credibility assessment; online tools • IT • Centralized – Standards and supported platforms • Decentralized – Local Support Providers • Usability analysis • Stewardship • Professional Development (CTL) • Workshops; consulting; faculty fellows

36. Other Useful Concepts • Outsourcing • Noel-Levitz; College Board • NSSE; CSRDE; Delaware • Structural arrangements • Split positions; hosted staff; exchanges • Contracted services

37. Still Other Useful Concepts • Networking Structures • Coordinating committees • IR Council • Quality facilitators • Brokering • Linking to expert resources • Internal and external

38. Summary • We are victims of our own success • We da man / We go, girl • The more we do to respond to demand, the more demand we create • We must go from providing fish to teaching people how to fish • We must learn together with our colleagues how to improve our fishing yield

39. Implications • We need to expand our ideas about deploying IR products and services • More emphasis on coordination and collaboration • We need to develop ourselves as more informed and expansive higher education administrators • What doesn’t kill us will surely make us stronger

40. Good to Great (Social Sector) Jim Collins

41. What is IR? • What are we good at? • Figure out how to cull use evidence from the information droppings all around us • What are we passionate about? • Getting people to use evidence… • What do we have resources to support • By ourselves, not very much • With our colleagues, an H-E- of a lot more