Assessment

1. Assessment A New Way of Thinking and Seeing Things Peter Hernon, MCCLPHEI, July 2007

2. Assessment Contributing to student “success” Focus on learning more than “imparting” How do students learn? Information Literacy ACRL competencies http://www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm A FEW Issues Facing Academic Libraries Peter Hernon, MCCLPHEI, July 2007

3. Quotation • “Assessment and evaluation are intended as means to document educational quality and institutional effectiveness, foster institutional improvement, and demonstrate accountability. • Unfortunately, without a clear sense of the purpose for assessment, knowledge of what is to be assessed, and understanding how the results will be used, assessment efforts, Peter Hernon, MCCLPHEI, July 2007

4. Quotation (continued) all too often, become an end in themselves. • If the results of assessment are not used for information planning and decisions, colleges and universities often find themselves in positions of being data rich and information poor.” • Ronald L. Baker, Outcomes Assessment in Higher Education, p. 12. Peter Hernon, MCCLPHEI, July 2007

5. NEASC STANDARD SEVEN: Library and Information Resources • 1: The institution makes available the library and information resources necessary for the fulfillment of its mission and purposes. These resources support the academic and research program and the intellectual and cultural development of students, faculty, and staff. Library and information resources may include the holdings and necessary services and equipment of libraries, media centers, computer centers, language laboratories, museums, and any other repositories of information or technological systems required for the support of institutional offerings. Clear and disseminated policies govern access, usage, and maintenance of the library, information resources, and services. The institution ensures that students use these resources as an integral part of their education. • 2: Through the institution's ownership or guaranteed access, sufficient collections, information technology systems, and services are readily accessible to students wherever programs are located or however they are delivered. These collections, systems, and services are sufficient in quality, level, diversity, quantity, and currency to support and enrich the institution's academic offerings. The institution provides facilities adequate to house the collections and equipment so as to foster an atmosphere conducive to inquiry, study, and learning among students, faculty, and staff. Peter Hernon, MCCLPHEI, July 2007

6. NEASC STANDARD SEVEN (continued) • 3: The institution provides sufficient and consistent financial support for the effective maintenance and improvement of the institution's library, information resources, and services. It makes provision for their proper maintenance, preservation, currency, and security. It allocates resources for scholarly support services compatible with its instructional and research programs and the needs of faculty and students. • 4: Professionally qualified and numerically adequate staff administer the institution's library, information resources, and services. The institution provides appropriate orientation and training for use of these resources, as well as instruction in basic information literacy. • 5: The institution participates in the exchange of resources and services with other institutions and within networks as necessary to support and supplement its educational programs. It provides appropriate support for distance learning students and faculty, such as on-line reference service and contractual access to relevant off-campus library resources. • 6: The institution regularly and systematically evaluates the adequacy and utilization of its library, information resources, and services and uses the results of the data to improve and increase the effectiveness of these services Peter Hernon, MCCLPHEI, July 2007

7. NEASC STANDARD Four:Programs and Instruction • Undergraduate Degree Programs • Graduate Degree Programs • Scholarship and Research • Instruction • Admissions and Retention • http://community.pmc.edu/neasc/standards/Default.htm Peter Hernon, MCCLPHEI, July 2007

8. Student outcomes are aggregate statistics on groups of students (e.g., graduation rates, retention rates, transfer rates, and employment rates for a graduating class) Such outcomes are institutional outcomes and are used to compare institutional performance They do not measure changes in students themselves due to their college experience these outcomes are outputs and reflect what the institution has accomplished; they do not reflect what (or how much) students learned Student learning outcomes —development of students: demonstrable acquisition of specific knowledge and skills How well do students Transfer and apply concepts, principles, ways of knowing, and problem solving across their major program of study? Integrate their core curriculum, general studies, or liberal studies into their major program or field of study? Develop understanding, behaviors, attitudes, values, and dispositions that the institution asserts it develops?* * Maki (2004) Student Outcomes versus Student Learning Outcomes Peter Hernon, MCCLPHEI, July 2007

9. Overview Questions—Does the Library Have a Role to Play? • What should students learn? • How does the content of one course relate to another? • How well are they learning it? • Across courses • What evidence do we gather to answer these questions? • Beyond graded assignments, course evaluations • How do we use that evidence to improve learning • Looking a courses as comprising a program Peter Hernon, MCCLPHEI, July 2007

10. Frameworks • Course Evaluation • Grades • Student course • evaluation • Course Assessment • Formative • Summative • Program Assessment • Program as a set of courses – • with some interconnection Peter Hernon, MCCLPHEI, July 2007

11. Examples of Different Perspectives Taxpayers: Cost, efficiency, affordability Peter Hernon, MCCLPHEI, July 2007

12. Evaluation Defined • The process of identifying and collecting data about specific services or activities, establishing criteria by which their success can be assessed, and determining both the quality of the service or activity and the degree to which the service or activity accomplishes stated goals and objectives. Peter Hernon, MCCLPHEI, July 2007

13. Assessment Defined • The ongoing process of: • Establishing clear, measurable expected outcomes of student learning • Ensuring that students have sufficient opportunities to achieve those outcomes • Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations • Using the resulting information to understand and improve student learning [at both the course and program] level Suskie (2004, p. 3) Peter Hernon, MCCLPHEI, July 2007

14. Improving student learning Work collaboratively: thinking beyond the set of courses you teach Accountability and accreditation— Meeting institutional mission Your mission is … Demonstrating institutional effectiveness Purposes of Assessment Peter Hernon, MCCLPHEI, July 2007

15. Institutional Effectiveness • Defined as how well an institutions achieves its mission and major institutional goals • “Since student learning is the heart of most institutional missions, the assessment of student learning is a major component of the assessment of institutional effectiveness.” • However, institutional effectiveness examines other aspects such as scholarship and research, community service, etc. • Suskie, 9-10 Peter Hernon, MCCLPHEI, July 2007

16. Peggy I. Maki defines learning as • Knowledge leading to understanding but also • Abilities • Habits of mind • Ways of knowing • Attitudes • Values • Other dispositions that an institution and its programs and services assert they develop. (p. 3) Peter Hernon, MCCLPHEI, July 2007

17. Student Learning Outcomes Skills Oral/written communication Foreign language communication Technological sophistication Quantitative reasoning ability Other Conceptual Leadership Critical Thinking Problem Solving Information Literacy Global Citizen Values (moral, etc.) Do “satisfaction” and “service quality” impact performance? Peter Hernon, MCCLPHEI, July 2007

18. What Is a Rubric? • A scoring guide: a list, chart, or guide that describes a set of the criteria (with levels of achievement along explicit dimensions) that enable “us” to score or interpret learning as students progress through the program • Perhaps using outside raters • A rubric translates a statement of a student learning into that set of criteria • Maki, Chapter 5; Hernon and Dugan, 274-88 Peter Hernon, MCCLPHEI, July 2007

19. Analyze Apply Demonstrate Develop Differentiate Identify Recognize Synthesize NOT Discuss Understand Etc. Verb Choices for Phrasing Student Learning Outcomes Peter Hernon, MCCLPHEI, July 2007

20. Examples of Student Learning Outcomes • Students will think logically and critical in solving problems; be able to evaluate, critique, and apply the thinking of others: and reach an appropriate conclusion. Novice, Intermediate, Advanced • Students will demonstrate leadership abilities (Leadership abilities evidenced in learning contexts include team building and shared decision making, strategic planning, advocacy, and consensus building and collaboration) Novice, Intermediate, Advanced Peter Hernon, MCCLPHEI, July 2007

21. Embedded course assessment (performance on assignments, etc.; minute paper) Portfolio assessment Performance (internships, practicum, student teaching) Professional jurors or evaluators Capstone course/experience Experimental research designs), with pre- and post-testing Use of standardized tests Think-aloud protocol Directed conversation Videotape/audiotape evaluation Analysis of theses/dissertations/ senior papers (content analysis, interviews, or oral defense) Direct Methods Peter Hernon, MCCLPHEI, July 2007

22. Surveys (self-reporting) and self-assessments Curriculum and syllabus evaluation Exit interviews Observation Other Indirect Methods Peter Hernon, MCCLPHEI, July 2007

23. Summary: Outcomes Assessment • Standard 4: student outcomes and student learning outcomes • Student learning outcomes measures changes in library users as a result of their contact with the library’s programs, resources and students • Institutional and program improvement, not external benchmarking; however, might be of some interest in “best practices” Peter Hernon, MCCLPHEI, July 2007

24. Any questions or issues I can help with today OR… Peter Hernon, MCCLPHEI, July 2007

25. Bibliography Hernon, P., & Dugan, R. E.(2004). Outcomes Assessment in Higher Education: Views and Perspectives. Westport, CT: Libraries Unlimited. Hernon, P., Dugan, R.E., & Schwartz, C. (2006). Revisiting Outcomes Assessment in Higher Education. Westport, CT: Libraries Unlimited. Hernon, P., & Calvert, P. Improving the Quality of Library Services for Students with Disabilities. Westport, CT: Libraries Unlimited. Maki, P. L. (2004). Assessing for Learning: Building a Sustainable Commitment across the Institution. Sterling, VA: Stylus. Maki, P. L., & Borkowski, N. A.(2006). The Assessment of Doctoral Education: Emerging Criteria and New Models for Improving Outcomes. Sterling, VA: Stylus. Suskie, L. (2004). Assessing Student Learning: A Common Sense Guide. Bolton, MA: Anker. Peter Hernon, MCCLPHEI, July 2007