1 / 25

Rubric-based Assessment of Information Literacy in Student Papers

Learn about the process of assessing information literacy skills in student papers using rubrics. Discover the challenges, trends, and potential improvements in information literacy assessment.

aldoa
Download Presentation

Rubric-based Assessment of Information Literacy in Student Papers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Imperfect but Authentic and Invaluable Information Literacy Assessment by Applying Rubrics to Anonymized Student Papers Dana Longley Assistant Director for Library Instruction and Information Literacy Slides: 

  2. Not Your Typical College • Non-traditional students • 18,600+ primarily part-time, working adults (ave age: 35) • 50% blended, 50% fully online • 7 overlapping terms schedule • No “campus” (35 regional centers in NYS + overseas) • Highly individualized study • Customizable degree programs • One-on-one mentoring • Narrative feedback (tests not generally used)  • No required courses

  3. Online Library • Small staff (4 PEs) • No director, student workers, or support staff • Fully online collections (no physical library) • No student info lit requirements (infused Info Mgt Gen Ed)

  4. Learning Assessment Timeline • 1980s: Area of Study reviews (degree portfolios) • 2002-03: SUNY-wide Gen. Ed. assessments • 2007-08: Assessments in the Major • 2012: College Learning Goals Policy drafted. Includes: • “Information and Digital Media Literacy: Critically access, evaluate, understand, create and share information using a range of collaborative technologies to advance learning, as well as personal and professional development.“

  5. Info Lit Assessments Official: • Gen. Ed. Info Management: via sample of student papers Unofficial: • Assessment in the Major: student papers • Synchronous Library Workshops: pre- & post-tests (voluntary) • Research Skills self-paced tutorial (voluntary) • 2 credit Info Lit course (voluntary)

  6. Information Management Competency: Ability to locate and use information from a variety of sources Outcomes: • Determine the quality of information needed • Locate an appropriate variety of information • Evaluate information and resources critically • Use information effectively to accomplish a specific purpose • Attribute external information sources and ideas correctly

  7. Pre-Assessment Process • Managed/owned by Office of Decision Support • Appropriate advanced level courses and assignments ID’d by faculty • Assessment team volunteers sought and verified • Papers + assignment instructions put into Adobe and all personal info scrubbed • Team reviews/discusses rubric and conducts norming session

  8. Team Assessment Sessions 2-day formal assessment session:  • 4-6 assessors • Most samples rated by 2 members  • Ratings input into Survey Monkey  • Analysis & inter-rater reliability via SPSS • Team discusses follow-up recommendations • Decision Support crafts report • Report shared internally

  9. Pros and Cons? Pros: • Authentic student work • Faculty and librarian input Cons: • Workload intensive when done • Uses online courses only (not representative?) • Some assignments don't fit rubric well • Small team sizes  • Subjectivity of rubric interpretation & scores? • No formal mechanism for implementing recommendations

  10. Results - Trends

  11. More Trends

  12. More Trends

  13. Possible broad conclusions? • Not great! • >50% samples = “does not meet” or “approaches” • Perception of info lit concepts growing among faculty? • Students have trouble especially with: • Synthesis • Evaluation • Attribution

  14. Quality

  15. Variety

  16. Evaluation

  17. Use

  18. Attribute

  19. 2016 Results by Outcome

  20. 2016 Results by Score

  21. Other IL-related Assessments

  22. Credit Course Data Self-assessment quiz at start: • Students struggle with: • Topic formation • ID of non-standard citations • Search strings • Understanding what lit reviews and primary sources are • In-text citations • Terminology (library jargon): journal vs journal article; database • Students already have some knowledge of: • ID of journal & book citations • Peer-review • Plagiarism basics • Basic publication types

  23. Looking Forward Ongoing or near future: • Update Information Management rubric • Better ID of relevant research assignments • Info Lit self-assessment in online orientation • Increased faculty awareness of IL assessment results & course and curricula remediation options Longer term: • Similar paper assessment between 4 yr periods (by library?)

  24. Questions? • Slides:  dana.longley@esc.edu @disobedientlib (twitter)

More Related