IL Best Practices:

1. IL Best Practices: Innovative Approaches to Fit Your Institutional Needs Sheril Hook, Instruction Coordinator, University of Toronto Mississauga Stephanie Sterling Brasley, Manager, Information Literacy Initiatives, California State University, Office of the Chancellor

2. Participants will be able to • Articulate the major ideas from categories 5 and 10 of the Best Practices Guidelines • Identify assessment tools for information literacy development • Identify basic strategies for how categories 5 and 10 of the Best Practices Guidelines can be applied to their own instructional and institutional environments

3. Agenda • Category 5: articulation with the curriculum • Playing with the IL Standards • IL by design: embedding IL into course outcomes • Category 10: assessment and evaluation • Instruction statistics • Horizontal and vertical integration

4. ALA/ACRL Characteristics of Programs of Information Literacy that Illustrate Best Practices Category 5: Articulation with the Curriculum Articulation with the curriculum for an information literacy program: • is formalized and widely disseminated; • emphasizes student-centered learning; • uses local governance structures to ensure institution-wide integration into academic or vocational programs; • identifies the scope (i.e., depth and complexity) of competencies to be acquired on a disciplinary level as well as at the course level; • sequences and integrates competencies throughout a student’s academic career, progressing in sophistication; and • specifies programs and courses charged with implementation. http://www.ala.org/ala/acrl/acrlstandards/characteristics.htm

5. IL Standards Standard One The information literate student determines the nature and extent of the information Performance Indicator 2. The information literate student identifies a variety of types and formats of potential sources for information. Outcomes include • Knows how information is formally and informally produced, organized, and disseminated • Recognizes that knowledge can be organized into disciplines that influence the way information is accessed • Identifies the value and differences of potential resources in a variety of formats (e.g., multimedia, database, website, data set, audio/visual, book) • Differentiates between primary and secondary sources, recognizing how their use and importance vary with each discipline • Realizes that information may need to be constructed with raw data from primary sources "Information Literacy Competency Standards for Higher Education." American Library Association. 2006.http://www.ala.org/acrl/ilcomstan.html (Accessed 15 May, 2007)

6. Let’s play with the standards

7. IL Standards Standard One The information literate student determines the nature and extent of the information Performance Indicator 2. The information literate student _________ a variety of types and formats of potential sources for information. Outcomes include • ________ information is formally and informally produced, organized, and disseminated • _________ knowledge can be organized into disciplines that influence the way information is accessed • __________the value and differences of potential resources in a variety of formats (e.g., multimedia, database, website, data set, audio/visual, book) • __________between primary and secondary sources, recognizing how their use and importance vary with each discipline • __________ that information may need to be constructed with raw data from primary sources "Information Literacy Competency Standards for Higher Education." American Library Association. 2006.http://www.ala.org/acrl/ilcomstan.html (Accessed 15 May, 2007)

8. Student Engagement:IL depth & complexity • research-based learning • problem-based learning • inquiry-based learning • case-based learning • discovery learning knowledge building Scardamalia, M., & Bereiter, C. (2003).

9. Research Question (inquiry-based) • How have myths changed over time? Humanities 2.

10. Assignment:Myth over Time • Outcomes • Explore the dynamism of myth by comparing and contrasting a selection of ancient and modern primary sources of a myth (at least one literary, one material) • Identify the most significant changes from ancient to modern source and discuss those changes in light of the context in which each source was created • Interpret those changes in terms of how they affect the meaning of the myth and how they came about in the first place Humanities 1.

11. Outcomes • compare and contrast a selection • of primary sources (art) • Students begin by finding primary sources--art works, music, scripts, opera and background information on artists Google has images, but no provenance information Camio has images, plus provenance and usage rights information Humanities 3.

12. Outcomes • identify the most significant changes...in light of the context in which each source was created. Students build on the learning acquired by finding background information on a time period/place Humanities 4.

13. Outcomes • identify the most significant changes...in light of the context in which each source was created. Students place a myth in the cultural context in which it’s being used or re-told Humanities 5.

14. Outcomes • compare and contrast a selection of primary sources (music) Students listen to a symphony to identify the dynamism of the myth and interpret its significance Humanities 6.

15. ALA/ACRL Characteristics of Programs of Information Literacy that Illustrate Best Practices Category 10: Assessment/Evaluation Assessment/evaluation of information literacy includes program performance and student outcomes and: for program evaluation: • establishes the process of ongoing planning/improvement of the program; • measures directly progress toward meeting the goals and objectives of the program; • integrates with course and curriculum assessment as well as institutional evaluations and regional/professional accreditation initiatives; and • assumes multiple methods and purposes for assessment/evaluation-- formative and summative-- short term and longitudinal; http://www.ala.org/ala/acrl/acrlstandards/characteristics.htm

16. ALA/ACRL Characteristics of Programs of Information Literacy that Illustrate Best Practices Category 10: Assessment/Evaluation (cont’d) Assessment/evaluation of information literacy includes program performance and student outcomes and: for student outcomes: • acknowledges differences in learning and teaching styles by using a variety of appropriate outcome measures, such as portfolio assessment, oral defense, quizzes, essays, direct observation, anecdotal, peer and self review, and experience; • focuses on student performance, knowledge acquisition, and attitude appraisal; • assesses both process and product; • includes student-, peer-, and self-evaluation; http://www.ala.org/ala/acrl/acrlstandards/characteristics.htm

17. Assessment Assessment is the process of gathering and discussing information for multiple and diverse purposes in order to develop a deep understanding of what students know, understand, and can do with their knowledge as a result of their educational experiences; the process culminates when assessment results are used to improve subsequent learning. Mary E. Huba and Jann E. Freed. Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning. Allyn & Bacon, 2000.

18. Evaluation Evaluation is “any effort to use assessment evidence to improve institutional, departmental, divisional, or institutional effectiveness”

19. Program Evaluation Components • An Assessment/Evaluation Program Plan • What are we assessing? • What are our IL instruction goals? • What are the measurable outcomes? • How will we measure at the course/curricular level? • Who are our campus partners? • How can we tie into institutional efforts? • What data do we need to collect?

20. Program Evaluation: Snapshot of Course Penetration • 100/200 level course penetration • Where • Core courses, instructor status • What • Standards 1-5 • How • Concept-based or tool-based

21. Instruction Database

22. Instruction Statistics

23. Stats tell a story • Redundancy • Gaps • Planning

24. Stats identify approaches Fig. 5 Number of unique instruction sessions given by type

25. Class snapshot

26. Class Snapshot, contd

27. Interpreting Statistics

28. Assessment of Student Outcomes

29. Definition Direct evidence of student performance collected from students Actual samples of student work Assess student performance by look at their work products Examples Assignments Research papers Portfolios Dissertations/theses Oral presentations Websites Posters/videos Instructor-designed exams or quizzes Direct Assessment

30. Definition Provides perspectives and perceptions about what has been learned Faculty must infer students’ skills, knowledge, and abilities rather than observing from direct evidence Examples: Exit interviews Surveys: student satisfaction, Course instruction Focus groups Self-reported reflections Research journals or diaries Student ratings of skills Graduation Rates Job Placement Rates Indirect Assessment

31. Formative Assessment • Ongoing measure of student learning • Provides feedback to student and instructor on learning process • Takes place during the learning process • Examples: • Classroom assessment techniques, targeted questions, in-class exercises, research journals/diaries, concept maps

32. Summative Assessment • Information gathered at the end of instruction • Used to evaluate the efficacy of the learning activity • Answers the question of whether learners learned what you had hoped they would • Typically quantitative: • Examples: test scores, letter grades, graduation rates

33. Formative vs. Summative “When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative.” - Robert Stakes http://library.cpmc.columbia.edu/cere/web/ACGME/doc/formative_summative.pdf

34. Classroom Assessment • Provides continuous flow of accurate information on student learning • Learner-centered • Teacher Directed • Mutually Beneficial • Formative • Context-specific • Ongoing • Rooted in Good teaching practice

35. Classroom Techniques-Examples • 1-minute paper • Muddiest Point • 1-sentence Summary • 3-2-1

36. CATS – Web Form Example http://www2.library.ucla.edu/libraries/college/11306.cfm One Minute Paper form • Quarter • Class • Instructor • Librarian • Your name • Your phone number • Your email address • 1. What is the most significant or meaningful thing you have learned during the session? (Feel free to add more items) • 2. What question(s) remain uppermost in your mind?

37. Example: Indiana University Bloomington Libraries • http://www.indiana.edu/~libinstr/Information_Literacy/assessment.html Basic Goal 1. Appreciates the richness and complexity of the information environment. Objective 1.1. You will be able to describe the wide array of information sources available and discuss their appropriateness for a given information problem. Possible Measurement Techniques: Essay examination Oral report Practicum in the library**Written evaluation assignment Basic Goal 4. Design and use a search strategy tailored to a specific information need. Objective 4.1. You will be able to describe and execute an appropriate search strategy in a given information source or in multiple information sources. Possible Measurement Techniques: Annotated bibliography with search strategy discussion included Collaborative learning exercise in class Practical exercise Practicum examination**Research journal Research paper proposal Research portfolio Research worksheet**

38. Diagnostic Assessment • Assesses knowledge and skills of students before instruction is designed • Determines where student is in terms of learning – identifying gaps in student learning • Can provide a baseline for student achievement in a subject area • Examples: standardized tests, review of students’ prior work, pre-tests/instructor-developed tests.

39. Assessment in Practice: iSkills, SAILS, iLit, ILT, ICDL

40. Assessment Tools: Evaluation Considerations • Audience • Test Development – Reliability and Validity • Costs: Development and Administration • Test Type: performance-based? Multiple-choice? • Delivery mechanism – web? Print? Simulations? • Content/Standards • Scoring and Reporting Features

41. SAILS – Standardized Assessment of Information Literacy Skills • Librarians at Kent State University, 2000 • Test of information literacy skills • Web-based test • Multiple choice questions • 142 whole items in American English • Students answer 40 items + 5 pilot • Based on 4 or 5 ACRL Standards (not 4)

42. iSkills Assessment (Formerly ICT Literacy Assessment) • ETS staff with CSU and 7 core institutions • Assesses Information and Communication Technology (ICT) Literacy • Web-based, performance-based, scenario-based test • Core and Advanced Versions; 75 minutes • 15 Tasks – 14 short (1-3 min) and 1 long (15 min) • Test tasks aligned with and informed by ACRL Standards, ISTE NETS standards

43. Information Literacy Test (ILT) • James Madison University’s Center for Assessment and Research Studies and the Library • Designed to assess the ACRL standards (not 4) • Web-based • Multiple Choice test • 60 operational items and 5 pilot • Password-protected-Secure administration • Reliability: 0.88 • Cost - ??

44. iLIT • Developed by CA. Community College librarians, CSU Subject Matter experts, Test Development experts, Psychometricians • Web-based • Multiple choice questions • Aligned to ACRL standards • “Affordable” • High-Stakes, proctored

45. Your Institution’s Needs The Characteristics in Practice