Outcome-Based Assessments Within the Context of Academic Libraries

1. Outcome-Based Assessments Within the Context of Academic Libraries Workshop: Day 2

2. Outcome-Based Assessments Within the Context of Academic Libraries • Acceptable Practices and Standards • Measuring/Demonstrating Learning Outcomes • Measuring/Demonstrating Other Outcomes/Impacts of Library Services • Qualitative Assessment Methods: Evaluation of Service Quality and Benchmarking • Making Effective Use of Available Data to Improve Library Services/Operations • Assessing the Value the Library Adds to the University Community

3. Outcome-Based Assessments Within the Context ofAcademic Libraries Acceptable Practices and Standards

4. Acceptable Practices and Standards • Key Concepts Related to Outcome-Based Assessments • Key Practices Identified in ACRL Standards & Guidelines • Desirable Practices • Essential Practices • Future Practices • Key Terms Defined – Workbook

5. Key Concept # 1Link Library Practices to the Institution • Strategic Planning Process • Institution Vision, Mission & Goals • New Course/Program Approval • Program Reviews by degree program, by professional organizations

6. Key Concept # 2Link Library Outcomes to Institution Outcomes • Identify institutional outcomes to which academiclibraries contribute: • Learning Outcomes • Other Outcomes

7. Key Concept # 3LinkLibrary Practices to Outcomes • Develop Library Instruction Program • Implement an Information Literacy Program • Develop a Library Liaison Program • Keep Pace With Technology:AccessServices • Use Comparisons:Peer Group & Longitudinal • Establish a Library Information Commons • Build an Innovative Learning Resource Center

8. Library Practice # 1 Develop Library Instruction Program • Library Classes & Workshops • First Year Experience (FYE) Program • New Faculty/Faculty Orientation • Online Instruction/Guides: • Distance Learners & Courses Online • Research Guides

9. Library Practice # 2Implement an Information Literacy Program: • General Curriculum Requirement • Discipline based skills • Lifelong competencies • Assessment of Student Learning

10. Library Practice # 3Develop a Library Liaison Program • Collaborate with Classroom Faculty • New Faculty/Faculty Orientation • Resource Guides that link to Resources • Services, Instruction & Guides Tailored to: • Specific Courses • Other Institution Outcomes

11. Library Practice # 4Keep Pace With Technology: Access Services • Wireless & PC Convertibles (PC Tablets) • Search Technologies:Metasearching, Federated Searching & SFX • Consortium Services: Access to Combined Catalogs • Electronic Reserves • Digital Libraries • Implications for Staffing & Training

12. Library Practice # 5:Use Comparisons: Peer Group & Longitudinal • Use Peer Groups to Compare Holdings, and access to full-text online: • Adequate support for specific programs • Tool: OCLC Collection Analysis Program • Use Longitudinal Comparisons to Identify Trends

13. Library Practice # 6Establish a Library Information Commons • What Makes an Information Commons Popular! • From Planning Steps to Implementation • Productivity Center for Presentation Applications • Implications for Staffing and Training

14. Future Library Practice # 1Build an Innovative Learning Resource Center • Next Step in Providing Seamless Support of Student Learning • Major Commitment by Institution: Strategic Plan, Funding, & Expected Outcomes to be Achieved • Joint Effort – Partnerships with Other Centers & Departments on Campus • #1 Student Priority – Georgia Tech’s Plans

15. Outcome-Based Assessments Within the Context of Academic Libraries Measuring/Demonstrating Learning Outcomes

16. Measuring/Demonstrating Learning Outcomes • Identify Library Learning Outcomes that Support Institution Learning Outcomes • Identify Library Practices that have Learning Outcomes: Information Literacy • Information Literacy (IL) expands typical instruction program to include assessment of student learning • IL program components are guided by an Instruction and Assessment Plan

17. Information Literacy: Learning Outcomes and Effectiveness • Examples of Student Learning Outcomes, Types of Performance-Based Measures & Assessment Techniques • Instruction Assessment Survey: Teaching and Instructional Materials Effectiveness • Knowledge Survey • Self-Assessment

18. National Level Efforts in United States • Association of Research Libraries • ARL New Measures Initiatives • Project SAILS (Standardized Assessment of Information Literacy Skills) • Educational Testing Service (ETS): • ICT Literacy Assessment

19. Other Resources: • Assessing Student Learning: Available Resources. C&RL News, May 2004 • Directory of Online Resources for Information Literacy • Planning, Implementing and Assessing Your Information Literacy Program

20. Outcome-Based Assessments Within the Context of Academic Libraries Measuring/Demonstrating OtherOutcomes/Impacts of Library Services

21. Measuring/Demonstrating Other Outcomes/Impacts of Library Services • KeyTerms: • Outcomes: the ways in which library users are changed as a result of their contact with the library's resources and programs [Learning Outcomes and Other Outcomes or Impacts] Task Force on Academic Library Outcomes Assessment Report, 1998 • Impacts: other contributions to institutional or campus-wide goals

22. Measuring/Demonstrating Other Outcomes/Impacts of Library Services • Assessment Diagram: Use the diagram to focus on Other Outcomes that measure Educational Impact • Accreditation Based Outcomes Chart • Expected Outcomes [illustrative examples] • Means for Assessment • Balanced Scorecard System from Business • Provides basis for executing a good strategy well • and managing change successfully

23. Balanced Scorecard System: Key Elements • Management system: • Clarifies vision & strategy, & translates into action • Incorporates Total Quality Management (TQM) • Provides a balance of different ways of measuring • Links an organization’s operations to its Strategy • Views the organization from 4 perspectives: • Customer [Students, Faculty and Staff] • Internal Business Processes • Learning and Growth • Financial [Retention, Employment, Graduate School]

24. More Balanced Scorecard: How It Works • Needs and expectations of customers & stakeholders • Strategy that defines your intended customer • Operations execute strategies and produce products and services for customers & stakeholders • Capabilities of a library’s organization and infrastructure enable its operations to efficiently satisfy customer and stakeholder requirements • Stakeholder contributions • Products and services provided to customers create desired outcomes: learning and other contributions

25. Measuring/Demonstrating Other Outcomes/Impacts of Library ServicesReview: Questions and Comments • Assessment Diagram: Use the diagram to focus on Other Outcomes that measure Educational Impact • Accreditation Based Outcomes Chart • Expected Outcomes [illustrative examples] • Means for Assessment • Balanced Scorecard System from Business • Provides basis for executing a good strategy well • and managing change successfully

26. Outcome-Based Assessments Within the Contextof Academic Libraries Qualitative Assessment Methods: Evaluation of Service Quality and Benchmarking

27. Qualitative Assessment Methods:Evaluation of Service Quality and Benchmarking • What These Methods Do: • Service Quality: Focus on the needs & expectations of customers to improve products & services • Benchmarking: Measure operational performance of an organization against external benchmarks to improve performance • Gauging the pulse of your library through service quality and benchmarking is an excellent first step in achieving and maintaining Excellence

28. Evaluation of Service Quality • Definition: Customers’ subjective evaluation of the quality of a service in relation to expectations • Why measure Service Quality vs Satisfaction? • Service Quality measures the gap between the customer’s level of expectation and how well the service was rated • Satisfaction measures how well you were satisfied or liked the service • Service Quality Example: LIBQUAL+ • Measures library users’ perceptions of service quality and identifies gaps between desired, perceived, and minimum expectations of service

29. Evaluation of Service Quality • LIBQUAL+ Program Goals: • Establish a library service quality assessment program at ARL • Develop web-based tools for assessing library service quality • Develop mechanisms and protocols for evaluating libraries • Identify best practices in providing library service • Benefits of Assessing Service Quality: • Identify where services need improvement in the view of users • Provide services that are more closely aligned with expectations • Compare your service quality with peer institutions in an effort to develop benchmarks and understanding of best practices

30. Evaluation of Service Quality • LIBQUAL+ covers 3 service quality dimensions: • Affect of Service • Library as Place • Information Control • Each question has 3 parts for respondents to indicate: • Minimum service level they will accept • Desired service level they expect • Perceived level of service currently provided • Design permits analysis of gaps between expectations, perception, and minimum acceptance level of service

31. Evaluation of Benchmarking • Definition: Process of identifying, sharing, and using best practices to improve business processes • Why Benchmarking Works: • It helps organizations exploit improvement efforts . . . . • It helps to blend continuous and breakthrough improvement . . . . • Key themes of successful benchmarking: willingness to learn; learning and transfer is an interactive, dynamic and ongoing process; transfer is a people-to-people process

32. Evaluation of Benchmarking • Benefits of Benchmarking: • Improve effectiveness and bottom line • Accelerate and manage change • Create a sense of urgency • Set stretch goals • Achieve breakthroughs and innovations • Quest for Excellence Examples: • Malcolm Baldrige National Quality Award

33. Outcome-Based Assessments Within the Context of Academic Libraries Making Effective Use of Available Data to Improve Services/Operations

34. Making Effective Use of Available Data to Improve Library Services/Operations • Identify data beingcollected on the library: • By the Library • Other Offices on Campus • External Organizations: ACRL, Peer Libraries • Purpose for collecting the data • Types of data and how is the data being used • Frequency that the data is being collected

35. Purpose and Types of Data Being Collected • User Surveys/Questionnaires: • Satisfaction with Library, Services, and Resources • Satisfaction with Library West Commons • Instruction Surveys • Service Quality • Identify Methods of Assessment Providing Evidence of Student Learning: • Direct Evidence • Indirect Evidence

36. Outcome-Based Assessments Within the Context of Academic Libraries Assessing the Value the Library Adds to the University Community

37. Assessing the Value the Library Adds to the University Community • Support of Institution/Campus-wide Goals and Objectives: 3 to 5 year Analysis • Demonstrated Support of Institution Vision • Learning Outcomes- Information & Computer Literacy • Other Outcomes/Impacts of Library Services • Support and Recognition of Faculty Research & Publication • Librarians on University Committees and support of campus & community events

38. Assessing the Value the Library Adds to the University Community • The Library Facility as a Place: • For 24 X 7 Learning – Recognized as Campus Center for Learning with librarians and trained staff to assist in locating, processing and presenting information • With individual/group study spaces & lab/classrooms equipped with latest technology and applications • That expresses the culture of its environment: displays of art work, exhibits and receptions calling attention to events & special topics of interest

39. Assessing the Value the Library Adds to the University Community • Achieving the Future Through Giving Opportunities: Support Your Library • Friends of the Library Events & Programs