Standardized Information Literacy Assessment: SAILS and Beyond

1. Standardized Information Literacy Assessment: SAILS and Beyond Juliet Rumble, Auburn UniversityCheryl Cecil & Beth Ashmore, Samford University Library April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

2. Project SAILSStandardized Assessment of Information Literacy Skills • An initiative of Kent State University Libraries • Purpose: develop a test instrument fortheassessment of information literacy skills that: --Is standardized --Assesses at institutional level --Provides for both external and internal benchmarking April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

3. Project Structure • 3 phase pilot testing of instrument (2003-2005) --Samford and AU participated in pilot testing --82 institutions participated (including 7 Canadian) --42,304 students tested • Diverse group of participating institutions --Carnegie Doctoral/Research level universities --2-4 year colleges April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

4. SAILS Test Instrument • Multiple choice test • 45 questions randomly drawn from a data bank of 252 items • Each test question addresses an ACRL Objective for Information Literacy Instruction April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

5. SAILS Administration at Samford • Sample: --mostly freshmen; students enrolled in UCCA 102 --approximately 40 juniors and senior from research methods and senior seminar courses • Campus stakeholders: --University Core Communication Arts Program --Office of Institutional Research and Assessment --Center for Teaching, Learning & Scholarship • Incentives: some instructors gave credit for taking the assessment; others required it; others just encouraged students to take it April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

6. SAILS administration at Auburn • Sample: --freshmen through seniors participated --students from 12 disciplines/subject majors • Campus stakeholders: --English Freshman Composition program --Office of Institutional Research and Assessment -- Core Curriculum Oversight Committee • Incentives: participants entered in drawing for 3 Apple iPods April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

7. SAILS Data Reports Results reported at two levels of specificity: --4 ACRL Info Lit Competency Standards --12 skill sets (derived from ACRL’s Objectives for Information Literacy Instruction) • Measures the performance of groups, not individuals • Test questions plotted according to difficulty level April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

8. What we learned from the SAILS data…. • On all standards and skill sets, the average student at Samford and Auburn scored at about the same level as the average student from all participating institutions. • Both Samford & Auburn focused on person-item maps for the 12 skill sets to identify areas of strength and weakness in students’ performance. April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

9. April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

10. A Difficult Question for Our Students If you have a research paper due, and the course instructor has not advised you to use a particular citation style, which of the following is the best thing to do? CHOOSE ONLY ONE ANSWER.􀀻 Select a citation style and use it consistently.􀂉 Use various citations styles based on the type of resource.􀂉 Use your own citation style and use it consistently.􀂉 You should always use APA if no other style is requested.􀂉 You should always use MLA if no other style is requested. April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

11. What the numbers don’t tell us…. • The national “benchmarks” associated with standards and skill sets do not indicate “mastery” of information literacy • Scores are not based on success in actually performing tasks associated with learning outcomes (although they are intended to be predictors of success). • Samford & Auburn’s test results do not track development of cohort groups (no longitudinal studies were conducted). • There’s still a lot we don’t know about cohort groups that we’re comparing. -- E.g.: Did test takers receive library instruction? -- E.g.: How much (and what kind of) instruction in research skills did students receive in other classes? April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006

12. What we learned about doing assessment • Sharing resources and expertise with other campus groups charged with programmatic assessment is a key to success. • Programmatic assessment involves a serious commitment of time and money. Support must come, not only from individual faculty members and departments, but also from university administration. • Assessment can be done without a standardized tool. In all likelihood, we need a variety of different assessment tools. --E.g. Pre- and post- tests that address specific learning objectives April 27, 2006 AACRL/CUS Best Practices Alabama Library Association Annual Conference 2006