Assessment of Information Literacy: Implementation Strategies

1. Assessment of Information Literacy: Implementation Strategies Deborah Gaspar Laura Maldonado Jennifer Nutefall George Washington University MLA May 4, 2006

2. Big Picture Academic / Schools Middle States University Guidelines State/Legislative measures Prove importance to university/school Public Board of Trustees Community Small Picture Improve teaching Evaluate programs Measure student achievement Demonstrate budgetary value Improve in-person/VR interactions Why assess? MLA May 4, 2006

3. Defining assessment • Write down your definition of assessment MLA May 4, 2006

4. Defining assessment • Formative - The gathering of information about student learning during the progression of a course or program to improve the learning of those students. • Summative - The gathering of information at the conclusion of a course, program, or undergraduate career to improve learning or to meet accountability demands. When used for improvement, impacts the next cohort of students taking the course or program. MLA May 4, 2006

5. Why are you assessing? • Write down why your library/institution is starting an assessment project (alternate: Why did you decide to attend today’s session?) • At your table, discuss what is currently happening at your institution in relation to assessment MLA May 4, 2006

6. Assessment data types • Quantitative – Summarized using numerical data analyzed mathematically and often reported as statistics. Examples: circulation or usage statistics, attendance figures. • Qualitative - Data is collected and reported using rich description of people, places, and specific contexts. Examples: ethnography, case studies, discourse analysis. MLA May 4, 2006

7. Before you start assessing • Get buy-in from staff • Decide on a common definition of assessment • Create learning outcomes • For instruction sessions • For reference interviews • Talk to others on campus about assessment • Plan to use multiple assessment tools MLA May 4, 2006

8. Five questions for instructional design • What do you want the patron to be able to do? • What does the patron need to know in order to do this well? • What activity will facilitate the learning? • How will the patron demonstrate the learning? • How will I know the patron has done this well? MLA May 4, 2006

9. Learning outcomes Assessment • Good outcomes are: • Action oriented • Measurable • Transferable • Clear to the student • “in order to” • Reflective of Bloom’s Taxonomy MLA May 4, 2006

10. Learning outcome examples • By the end of class, students will be able to find background information on their conflict in order to choose a specific aspect to focus their research. • By the end of class, students will be able to break apart a research topic into major concepts in order to formulate a search strategy. MLA May 4, 2006

11. Learning outcome examples • By the end of the reference interview, the patron will be able to access an Ebsco database in order to locate a needed article. • By the end of the reference interview, the patron will be able to locate information about library programs in order to participate. MLA May 4, 2006

12. Learning outcome exercise • Read information provided • List what should be covered in a library instruction class in order for the students to complete the assignment • What are the 3 most important things to be covered in the instruction? • Write one learning outcome MLA May 4, 2006

13. Cycle of assessment at GW • Beginning stages • Creation of different types of assessment • Use of an assessment portfolio • Recent accomplishments • Involvement with the University Writing Program • Inclusion of questions on course evaluations for freshman writing course (UW20) • Assessment group MLA May 4, 2006

14. Beginning stages: Types of assessment • Research Experience • One Minute Paper • Faculty Assessment • Observation Assessment • Research Journal MLA May 4, 2006

15. Hand out on the first day of class Students answer two questions In your previous research experience, where were you most likely to look for information? What are some research difficulties and/or successes you’ve experienced? Sample comments: It takes me awhile to weed through useless information Online: just google it and hope for the best Either unable to find any information pertaining to my topic or finding too much to sort through online A research difficulty would be correctly documenting my sources Research experience MLA May 4, 2006

16. Provide general information with possible questions to ask Each instruction librarian chooses the questions to ask the students Librarian transcribes comments onto the worksheet Sample comments: Student wrote: “I did not know how to check and see if an article was in the library – I thought if only a citation was given, it was not findable. But now I know that is not true!” Librarian reflection: “I would spend time on constructing a search statement and allow for some brainstorming of ideas.” One minute paper MLA May 4, 2006

17. Asked faculty if okay to send them the form Librarian filled in learning outcomes Sent by email after the students have turned in their assignment Gave a deadline for the form to be returned Sample comments: “Without the session my students would have been relying totally on internet sources and books that I had recommended to them. The ability to search for journal articles and books is essential to their being able to get a sense of what has been written in a certain field, which was a core part of the class.” “Yes, I personally learned an enormous amount about the different databases and how to narrow and limit the number of articles that come up after plugging in the key words.” Initial faculty assessment MLA May 4, 2006

18. Observation assessment • Librarian deliberately looks at individual student actions or behavior to obtain assessment information • Sample librarian comments: • “They seemed to be able to use the keyword to subject heading concept – it was demonstrated in both WRLC Catalog, EbscoHost and Periodical Abstracts.” • “They had a harder time knowing when to stick with a keyword search and use Boolean “or” and “and” to get results when the subject heading search strategy was not working.” MLA May 4, 2006

19. Research journal • Annotated bibliography/research log OR a narrative essay based on a research log • Includes a reflection on the research process • Graded by both the professor and librarian MLA May 4, 2006

20. Beginning stages: Assessment portfolio • Purpose • Gather and present data and evidence related to teaching quality and student learning • Create an environment for librarians to think critically about teaching techniques, learning styles and evaluation of success • Share the teaching experience with new librarians • Function of professional development and self-reflection MLA May 4, 2006

21. Portfolio Table of Contents • Mission statement • Annual report • Assessment Plan • Assessment documents • Letters and Emails • Faculty Assessment • Observation Assessment • One minute paper worksheets • Attitudinal Assessment MLA May 4, 2006

22. Creating an assessment portfolio • Create a common understanding of assessment • Start incorporating assessment into instruction • Start small • Add information as it presents itself • Keep it in a location where it can be accessed and seen by many MLA May 4, 2006

23. Recent accomplishments • Course evaluations • Office of University Assessment added questions related to Information Literacy learning • Creation of assessment group • Revised faculty assessment • Peer assessment pilot project MLA May 4, 2006

24. Course evaluations • Worked with the Director for Academic Planning and Assessment • Questions added to University Writing 20 course evaluations (2003-2004) • UW20 improved my research skills • I feel comfortable using the GW libraries for research • I know how to find subject-specific scholarly articles • Having a librarian involved with UW20 enhanced the course experience • The class instruction session on library and research skills led by the class librarian was valuable • The research skills learned in UW20 were useful for other classes this semester MLA May 4, 2006

25. Assessment group • Why it was created • Discussion of current projects • Revised faculty assessment • Peer assessment pilot project • Future directions MLA May 4, 2006

26. Faculty survey • Limited number of questions • Anonymous • Online submission • Targeted feedback on library sessions • Faculty learning goals • Student follow-up questions • Student research in final papers • Effectiveness / Timing of library sessions MLA May 4, 2006

27. Peer assessment project • Conducted research of existing literature • Designed forms and format • Initiated by an instruction librarian • Selects an observer • Specifies targeted behavior or lesson plan • Form provides outline for observer • For the observation • For feedback • Self reflection form structures synthesis process for observed librarian MLA May 4, 2006

28. Assessment activity • As a group, take the learning outcome you created earlier • Design 2 distinct assessment tools, including questions • Keep in mind: • How will the patron demonstrate the learning? • How will I know the patron has done this well? • Present to the group • How will you use this assessment at your home institution? MLA May 4, 2006

29. Publishing / Reporting results • Select format based on target audience • Emphasize information most useful to audience • Provide information to all stakeholders • Clear, concise, informative MLA May 4, 2006

30. 3-2-1 results and discussion List 3 things you know about assessment: __________________________________ List 2 things you want to know about assessment: ___________________________________ List 1 thing your institution is doing with assessment: ___________________________________ MLA May 4, 2006

31. Questions? MLA May 4, 2006

32. Contact information • Jennifer Nutefall, Instruction Coordinator • jnutefal@gwu.edu • 202.994.9863 • Laura Maldonado, Instructional Librarian • maldonal@gwu.edu • 202.994.1373 • Deborah Gaspar, Instructional Librarian • dgaspar@gwu.edu • 202.994.1333 MLA May 4, 2006