1 / 43

Rubric Assessment of Student Responses to an Information Literacy Tutorial

Rubric Assessment of Student Responses to an Information Literacy Tutorial. Megan Oakleaf Librarian for Instruction & Undergraduate Research Steve McCann NCSU Libraries Fellow. Share our experiences. In changing the way we assess our program In adapting ACRL outcomes to our project

arnaud
Download Presentation

Rubric Assessment of Student Responses to an Information Literacy Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rubric Assessment of Student Responses to an Information Literacy Tutorial Megan Oakleaf Librarian for Instruction & Undergraduate Research Steve McCann NCSU Libraries Fellow

  2. Share our experiences. In changing the way we assess our program In adapting ACRL outcomes to our project In selecting a learning artifact to assess In piloting our assessment plan Facilitate your experience with this type of assessment. Provide “take away” ideas. Presentation Objectives

  3. NCSU LibrariesInstruction Program

  4. Information literacy is a set of abilities requiring individuals to "recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information." Association of College and Research Libraries http://www.ala.org/ala/acrl/acrlstandards /informationliteracycompetency.htm Information Literacy

  5. How have we measured our success?

  6. Goal: To assess student learning of information literacy skills using outcomes-based assessment. Need 2 things… An Artifact to Assess Outcomes to Measure The Decision to Change

  7. Interactions with students that could yield assessment artifacts… 50-minute one-shot workshops Library Online Basic Orientation (LOBO) What artifact will we assess?

  8. Perceived lack of time for open-ended responses, only m.choice and T/F are options. Taught by numerous librarians who lack assessment knowledge. Inconsistent audiences & content. Incomplete spectrum of outcomes addressed. Disadvantages of Using Workshops for Assessment

  9. Forms basis for IL instruction at NCSU. Reaches virtually all incoming freshmen. Recently redesigned, includes open-ended questions. Captures student responses in a searchable database. Potential for rich data. Advantages of Using LOBO for Assessment

  10. Information Literacy Competency Standards for Higher Education 5 Standards 22 Performance Indicators 87 Outcomes What outcomes will we assess?

  11. Objectives for Information Literacy Instruction: A Model Statement for Academic Librarians Standards = 5 Perform. Indicators = 14 Outcomes = 35 “Bullets” = 133 What outcomes will we assess?

  12. LOBO Objectives & Outcomes Objectives = 5 Measurable Outcomes= 45 What outcomes will we assess?

  13. Set up DB to access student answers. Match outcomes to questions. How will we know the outcome’s been met? Beginning, Developing, Exemplary Pilot test a section of LOBO. First Steps

  14. Problem: Students use web sources for academic purposes without evaluating their quality. Are they duped by low-quality sites? Can we teach them to be more critical consumers of information? The Pilot Test

  15. What criteria are you looking for? What clues can you find? What specific example can you give from the web site at hand? Is the web site a good one for you to use? Evaluating Web Sites

  16. Evaluating Web Sites

  17. What we found out from 50 randomly selected student accounts … Pilot Test Results

  18. 1,830 total accounts new accounts 46% of total questions answered Evaluated 50 accounts Lobo Data—Jan-April, ‘04

  19. #3 “Evaluate Web Sites” questions: Locate a website Evaluate website’s authority Evaluate recency/currency Identify Bias/Point of View Objective 5: Evaluating Resources

  20. 4 points possible: Average score 2.7 Q1: Locate a Website Question Text: “Type the title and URL (web address) of the web site you will evaluate here:”

  21. 8 points possible: Average score 5.0 Q2: Evaluate a Website’s Authority Question Text: “Answer the questions above for the web site you're evaluating. Overall, does what you know about the URL of the web site indicate that it's a good resource? ” “Answer the questions above for the web site you're evaluating. Overall, does what you know about the authorship of the web site indicate that it's a good resource? ”

  22. 8 points possible: Average score 5.6 Q3: Evaluate a Website’s Currency Question Text: “Answer the questions above for the web site you're evaluating. Overall, does what you know about the currency of the web site indicate that it's a good resource? ”

  23. 8 points possible: Average score 4.1 Q4: Identify a Website’s Bias Question Text: “Answer the questions above for the web site you're evaluating. Overall, does what you know about the bias of the web site indicate that it's a good resource? Overall, is this web site a good resource to use for your assignment?”

  24. Students proven successful with mechanical tasks like checking currency and identifying URLs. Students are shown as “developing” with judgment tasks such as authority and bias. Bias is a potential target area. Pilot Test Findings

  25. Changes to LOBO Add and Reorganize Content Improve Question Format Enlarge Response Space Make Rubrics Available? Train Course Instructors How are we using our assessment results?

  26. Changes to LOBO Changes Coming Soon! Add Viewlet to Model Application of Content Replace Questions with Content • Answer these questions about the web site you’re evaluating in the space below: • Who created the site? What point of view do they represent? • What organizations support the site? What biases might they have? • Are links included that point to other viewpoints? • Are there signs of bias included in the site? • Are you biased toward the site? • Overall, does what you know about the bias of the web site indicate that it’s a good resource? Fix Questions, Add Links, Enlarge Answer Space Sample Student Answers Get Help with Your Answer

  27. Changes for Instructors Share Rubrics, Continue Ongoing Training, Add Lesson Plans

  28. Changes to the Instruction Program Including assessment in departmental 3-yr goals Sharing data with “subject-specialist” librarians Initiating rubric assessment of advanced instruction Reporting results to library administration and colleagues on other campuses How are we using our assessment results?

  29. Rubrics require lots of revision! Rubrics are effective in measuring higher-level thinking skills. Rubrics provide information administrators can use for reporting and instructors can use to improve teaching and learning. Our colleagues are interested in our progress.  Key Learning

  30. Can higher-level thinking skills like information literacy or critical thinking be adequately described in a rubric? Are rubrics good tools for assessing student responses to tutorials? What problems did you find in your practice with these rubrics? What part of this process/project could be applied at your institution? Take Aways

  31. Muddiest Points?

  32. Megan Oakleaf Librarian for Instruction & Undergraduate Research 919-513-0302 megan_oakleaf@ncsu.edu Steve McCann NCSU Libraries Fellow 919-513-7080 steve_mccann@ncsu.edu Contact Information

  33. Rubric Assessment of Student Responses to an Information Literacy Tutorial Megan Oakleaf Librarian for Instruction & Undergraduate Research Steve McCann NCSU Libraries Fellow

More Related