1 / 42

Assessment of Library Instruction

@ Randall Library. Assessment of Library Instruction. What is Assessment?. “Assessment is an ongoing process aimed at understanding and improving student learning.” 1 Main purposes: 2 Increase student learning Strengthen instructional program Answer calls for accountability.

cale
Download Presentation

Assessment of Library Instruction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. @ Randall Library Assessment ofLibrary Instruction

  2. What is Assessment? • “Assessment is an ongoing process aimed at understanding and improving student learning.”1 Main purposes: 2 • Increase student learning • Strengthen instructional program • Answer calls for accountability http://www.acrl.org/ala/mgrps/divs/acrl/issues/infolit/resources/assess/issues.cfm (1) Oakleaf & Kaske, 2009 (2)

  3. Why Assess Library Instruction? • To increase student learning • To improve instruction • To inform future assessment efforts • To provide data for accountability measures • To collaborate with faculty • To collaborate with other librarians • To argue for increased resources http://libraryassessment.org/bm~doc/oakleaf_assessment_cycle.pps Forward to Slide 9

  4. Why Assess Library Instruction? • “Well done assessment is essential to documenting the effects of information literacy programs and the degree to which they are meeting their objectives. The assessment process can also assist anyone involved in information literacy instruction by getting them actively involved in the process and engaged in classroom activities, by giving them feedback about students’ thinking and learning, and by helping them focus on goals and objectives …” Assessing Student Learning Outcomes for Information Literacy Instruction in Academic Institutions

  5. Resistance and Misperceptions • “_____ is making us do this.” • “_____ is trying to tell us how to teach.” • “We have to get this over with.” (Once and done) • “I don’t have time for this.” • “I have to develop a pre-test and a post-test.” • “Assessment is a fad.”

  6. What Needs to be Assessed? • Macro to micro • UNCW curriculum as a whole • All UNCW programs / majors • All UNCW courses • Current “Basic Studies” program / New “University Studies” program (General Education) • Information Literacy (IL) component of University Studies curriculum (as a whole) • “Information Literacy courses” within IL component • Library instruction program • Individual library instruction sessions

  7. Past Library Instruction Assessment • Library Instruction Sessions • Primary focus on teaching evaluation • Online student evaluation forms • Peer (librarian) observations • Faculty evaluations of teaching (Attending instructor) • Focus has been on us • Perceptions of teaching • In general, little measure of what students are actually learning / can do • Some measure of what they feel they are learning • Indirect assessment

  8. Moving Forward • Comprehensive library instruction assessment plan • Instruction program (holistically) • Student learning outcomes for individual instruction sessions • Ongoing, continuous process

  9. Strategic Plan for Assessment • Determine purpose(s) of assessment • Examine mission of the university, library, and instruction program • Examine structure of library instruction program • Examine and redefine program goals • Collect and examine relevant data / statistics related to program goals (e.g. How many students take ENG 201?) • Examine program resources • Examine / establish collaborations with other faculty • Establish aggressive yet attainable timeline • What will we assess? • How will we assess? • When will we assess? • What will we change because of assessment data?

  10. Purpose(s) • Why are we engaging in assessment? • What do we hope to gain from assessment efforts? Main purposes: • Increase student learning • Strengthen instructional program • Answer calls for accountability • See slide #3

  11. Other Purposes • Connection between library instruction assessment and UNCW assessment • Only area offering courses not represented in UNCW assessment efforts (?) • No representation on Learning Assessment Council • Instruction sessions a component of UNCW courses • Promotion of library instruction program and information literacy • Internally-focused purposes: • Improve workload, workflow, organizational structures in library, etc.

  12. Driving Forces (Randall Library) • Formation of Information Literacy Strategic Planning Team (Fall 2008) • Instructional Services Coordinator training: • ACRL Immersion Program (Summer 2008) • Master of Science in Instructional Technology (MIT) • MIT 531 (Assessment of Learning Outcomes) (Summer 2009) • LOEX Conference (May 2010) • NCLA Bibliographic Instruction Group: Assessment & Student Learning Outcomes Workshop (May 2010) • Georgia Conference on Information Literacy (October 2010) • Library Assessment Conference (October 2010)

  13. ILSPT • Mission Statement: “The Information Literacy Strategic Planning Team will leadlong-term, strategic planning to help transform the current set of instructional services activities into a well conceived, planned program that is integrated into both the basic studies curriculum and into each major. An information literacy program is one that utilizes well placed instruction sessions and tutorials/online modules to integrate information literacy skills into the curriculum of courses and builds skills progressively through a student’s college career.” • Add: Led by the Coordinator of Instructional Services, ILSPT will plan for and oversee the assessment of library instruction …

  14. Driving Forces (RL & UNCW) • UNCW SACS accreditation • New/revised “University Studies” program which includes Information Literacy Component • Need for library to demonstrate importance, effectiveness, need for resources, etc.

  15. Driving Forces (UNCW) • Highest priority: Student learning • UNCW shift to culture of assessment • 2005 campus assessment workshops (Susan Hatfield, consultant) • Learning Assessment Council • Institutional Research & Assessment • Assessment directors • Assessment of Basic Studies • National Survey of Student Engagement (NSSE) • Collegiate Learning Assessment (CLA) • General Education Assessment Committee (Report)

  16. Assessment so Far (UNCW) • UNCW Learning Goals • General Education Report (March 2009) • On Information Literacy • No librarians were consulted in the construction of this process / report • Instructional Services Coordinator contacted the Director of University Planning and sent the Information Literacy Competency Standards to Director through campus mail • Communication was not returned http://uncw.edu/assessment/Documents/General%20Education/GenEdAssessmentCommitteeReportMarch2009.pdf

  17. General Education Report • “All UNCW freshman seminar participants undergo instruction by the Randall Library in locating and using information, although there is no current assessment of student’s information literacy proficiency either before or after that instruction.”

  18. General Education Report • “Recent data indicate that college students have difficulty developing a topic question and are often not aware of informational resources available to them. Even when aware of appropriate information resources, they may lack the ability to adequately evaluate and synthesize information. This can result in classroom work based on haphazard discussion groups or personal opinion.” Albitz, R.S. (2007) The what and who of information literacy and critical thinking in higher education. Libraries and the Academy. 7, 1, 97‐109

  19. General Education Report • “Of 37 common learning outcomes established for the Basic Studies categories, only two (CMP3 and IP1A) fit into the category of information literacy. Perhaps there is not sufficient emphasis in the spectrum of Basic Studies courses on information literacy in a broad range of disciplines, or perhaps additional common learning outcomes should be developed to encompass efforts that are currently being made.” CMP3: “Employ a writing process that includes finding, evaluating, analyzing, and synthesizing appropriate primary and secondary sources, and successfully integrating personal knowledge with source material.” IP 1A: “Access and analyze information about topics or issues that are interdisciplinary or transdisciplinary.”

  20. National Survey of Student Engagement • “First-year students were asked in NSSE about the frequency with which they ‘worked on a paper or project that required integrating ideas or information from various sources,’ a component of information literacy. UNCW first-year students reported a frequency that was statistically significantly below that reported by our selected peers, significantly below that reported by national master’s universities, and significantly below that reported by all NSSE 2007 institutional participants. This information led to the development of a rubric-based assessment plan for information literacy to be implemented with the comprehensive assessment of Basic Studies beginning Fall 2009.” • Assessment tool selected: • The American Association of Colleges and Universities’ (AAC&U) “Information Literacy Metarubric” http://uncw.edu/assessment/BasicStudiesAssessment.html http://uncw.edu/assessment/Documents/General%20Education/2004-2005NSSE-BasicStudies.pdf

  21. Merging Efforts Information Literate UNCW Graduates UNCW (University Studies Assessment) Randall Library (Library Instruction Assessment)

  22. On the Horizon • University Studies (Fall 2011) • Students who complete the University Studies program will: • Goal 3. Locate, evaluate, and effectively use information by applying a variety of academic and technological skills (Information Literacy) http://www.uncw.edu/universitystudies/documents/Univ.StudiesCurriculumReport.pdf

  23. Information Literacy Curriculum • Component of University Studies curriculum • “Designed to foster the development of students’ information literacy skills by requiring three information literacy intensive courses: The First Year Experience and two additional information literacy intensive courses, with at least one in the major field of study.”

  24. Information Literacy Curriculum • Based on the Information Literacy Competency Standards for Higher Education • The information literate student … • Determines the nature and extent of the information needed. • Accesses needed information effectively and efficiently. • Evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system. • Individually or as a member of a group, uses information effectively to accomplish a specific purpose. • Understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.

  25. Information Literacy Curriculum • University Studies Information Literacy Student Learning Outcomes (adopted by Faculty Senate November 2009)The student will: • IL 1. Be able to determine the nature and extent of information needed to solve a problem. • IL 2. Access information effectively and efficiently from a variety of sources. • IL 3. Evaluate information critically and incorporate appropriate information into his or her knowledge base. • IL 4. Individually, or as a member of a group, use information effectively to accomplish a specific purpose. • IL 5. Understand many of the economic, legal, and social issues surrounding the use of information and access and use information ethically and legally.

  26. Information Literacy Curriculum Course in the Major (Research Methods or other)(3 hours) Library instruction sessions focus on IL Comp Standards 2 (Accessing info through discipline specific resources) and 5 (Ethical use: Discipline specific citations and writing) English Composition or other (e.g. LIB 103) (3 hours) Library instruction sessions focus on IL Comp Standards 1 (Determining info needed), 2 (Accessing info), and 4 (Using info effectively) Freshman Seminar (3 hours)Library instruction sessions focus on IL Comp Standards 3 (Evaluating information) and 5 (Ethical use: Plagiarism) 9 hours of courses (“Double dipping” from other components) Content reinforced throughout curriculum

  27. What’s Different for Us? • Every major must have an “information literacy intensive course” • Faculty must supply information literacy student learning outcomes for that course • Information literacy on the “radar” in a new way • “Just send ‘em to the library” does not apply • Potential for collaborations that did not exist before • Information Literacy is a goal of the entire University Studies program • The university will assess this

  28. Library Instruction Program Shift • From bibliographic instruction sessions to information literacy • Moving from scattered “one shot” instruction sessions to integration of information literacy skills instruction into UNCW curriculum • Defining the “Instruction Program” • Information literacy curriculum (University Studies) • Freshman Seminar instruction • English composition instruction (ENG 101, 103, and 201) • Instruction in each major • Workshops & tours • Online tutorials • LIB Courses • Regional engagement / instruction (high school student instruction, Pathways workshops, etc.)

  29. Current Library Instruction • AY 2008-2009 • Course-related instruction sessions, workshops, and tours • 587 sessions • 678 contact hours • Teaching 10,750 users PLUS • Thirty four hours of credit courses offered • LIB 101 (4 sections, 1credit hour, 19 students in each) • LIB 103 (9 sections, 3 credit hours, 19 students in each) • LIB 104 (1 section, 3 credit hours, 19 students in each) https://share.uncw.edu/sites/aa/library/pubsvcs/Shared%20Documents/Annual%20Reports/annrept0809.docx

  30. Moving to Assessment: A System • Part 1: Expected Outcomes (Part of Assessment Plan) • Part 2: Assessment Methods and Tools (Part of Assessment Plan) • Part 3: Implementation (Part of Assessment Plan) • Part 4: Summary of Findings (Part of Assessment Report) • Part 5: Actions Taken (Part of Assessment Report) http://www.uncw.edu/cas/assessmentplanandreport.html

  31. Steps of Assessment • Determine what we need to assess by creating Student Learning Outcomes for: • Freshman Seminar library instruction sessions • English Composition (ENG 101, 103, and 201) library instruction sessions • Instruction sessions in each major (as applicable) • LIB Courses • ILSPT has reviewed ACRL Standards • SLOs will “map” to ACRL Standards • Determine how we are going to assess • Means of Assessment (what method or tool we will use) • “My Research Strategy Worksheet” (scored using a rubric), sample of student products, online quiz, “5 minute paper,” tests, etc. • Determine what and when we will assess • Review assessment information • Make changes based on assessment information Repeat all steps continuously

  32. Assessment Cycle Library Instruction Program

  33. Assessment Checklist • What are our research questions? (What are we trying to discover about student skills, knowledge, abilities, etc.; and what evidence do we have already?) • What is the expected level of performance? • When in the students’ career do we assess this outcome? (entry, end of sophomore year, senior, etc.) • In which course(s) or venue? • What student work/artifacts are collected? • How is the student work evaluated? (criteria/rubric) • Who evaluates the student work? • Who analyzes the results? • Where do recommendations for action go? • Who takes action? (And how do we ensure changes are evidence‐based and data‐driven?) • How is the process documented? • Where is the documentation kept? • What is the timetable/schedule for determining which outcomes are assessed when? Developed by the General Education Assessment Committee for designing assessment of a learning outcome.

  34. Who Is Going to Do This? • All Instruction Librarians • Comprehensive plan coordinated by Instructional Services Coordinator • Supported by Information Literacy Strategic Planning Team • Assessment at micro level (instruction sessions) • Done by individual librarians with assistance from Instructional Services Coordinator • Will seek guidance from campus experts (e.g. Dr. Linda Seifert, Director of Assessment, College of Arts and Sciences)

  35. First Steps • Completion of assessment structure and plan outline (Summer 2010) • Creation of assessment website (Summer/Fall 2010) • LIB 103 assessment planning (Begin Fall 2010) • Freshman Seminar assessment planning • SLOs and means of assessment (Begin Fall 2010) • English Composition (ENG 101, 103, and 201) assessment planning • Draft of SLOs (Fall 2010) • Assessment of SLOs (Spring 2011) • Assessment planning for the majors • SLOs and means of assessment (Begin Fall 2010)

  36. Other Assessment & Evaluation • Assessment/Evaluation of: • Learners (SLOs) • Learning Opportunities • Learning Materials • Learning Environment • Instructors

  37. Assessment & Evaluation • SWOT/SOAR of library instruction program (Fall 2010) • Evaluation of library instruction program coordinator based on “Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators” (Fall 2010) • Self analysis of librarians based on “Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators” (Fall 2010) • Continuous use of online student evaluation forms • Continued annual peer assessment of teaching • Continued use of SPOTS for LIB courses • Future implementation of faculty surveys and evaluations • Review benchmarking data (Collected in fall 2008 and spring 2009) • Review of LibQual+ Survey results (Administered in 2008) • UNCW efforts: NSSE, CLA, General Education Assessment, etc.

  38. To Explore in the Future • Large Scale Information Literacy Assessment Tools for UNCW: • Project SAILS (Standardized Assessment of Information Literacy Skills) • Sample • James Madison Information Literacy Test (ILT) • iCritical Thinking™ Certification (ETS) • Replaced the ETS iSkills Test • Graduate Courses

  39. Data Management • WASSAIL (Open source) • “WASSAIL is a database-driven, web-based application employing PHP, MySQL, and Javascript/AJAX technologies. It was created to manage question and response data from the Augustana Library's library instruction sessions, pre- and post-tests from credit bearing information literacy (IL) courses, and user surveys. It has now expanded beyond its original function and is being used to manage question and  response data from a variety of settings.” • http://www.library.ualberta.ca/augustana/infolit/wassail/

  40. Helpful Resources • ACRL Information Literacy website (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm) • ACRL Institute for Information Literacy (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm) • ARL Library Assessment Conference (http://libraryassessment.org) • “A Twenty Year Path: Learning about Assessment; Learning from Assessment” (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER1/97) • Information Literacy in the Disciplines (http://wikis.ala.org/acrl/index.php/Information_literacy_in_the_disciplines)

  41. Helpful Resources • Internet Resources for Higher Education Outcomes Assessment (http://www2.acs.ncsu.edu/UPA/assmt/resource.htm) • Megan Oakleaf’s Presentations and Publications (http://meganoakleaf.info/presentations.html) • National Institute for Learning Outcomes Assessment (http://learningoutcomesassessment.org/) • Task Force on Academic Library Outcomes Assessment Report (http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/taskforceacademic.cfm) • Utah State University Library Instruction Assessment (http://library.usu.edu/instruct/assessment/index.php) • “Writing Information Literacy Assessment Plans: A Guide to Best Practice” (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER2/98)

  42. Library Instruction Assessment Website Anne Pemberton Instructional Services Coordinator Chair, Information Literacy Strategic Planning Team pembertona@uncw.edu http://library.uncw.edu/staff/anne_pemberton

More Related