160 likes | 182 Views
Explore levels of evaluation, outcomes-based strategies, & effective assessment in library user instruction. Learn to document & measure success through impactful methods.
E N D
BEST PRACTICES IN LIBRARY INSTRUCTION FORUMNovember 7, 2007 HOW DOES YOUR ASSESSMENT MEASURE UP? Karen Brown Dominican University
Goals: • Discuss different levels of evaluation • Examine the potential of outcomes-based evaluation • Apply outcomes-based strategies to library user instruction
Kirkpatrick’s Levels of Evaluation • LEVEL ONE: Reaction • LEVEL TWO: Learning • LEVEL THREE: Transfer • LEVEL FOUR: Results
LEVEL 1: REACTION • How did the participants react to the session?
LEVEL 2: LEARNING • Have the skills, knowledge, or attitudes of the participants increased, improved, or advanced?
LEVEL 3: BEHAVIOR (or transfer) • Do the participants use their newly acquired skills or knowledge in their jobs, courses, or personal endeavors?
LEVEL 4: RESULTS • What impact has the instruction achieved?
OUTCOMES-BASED EVALUATION – Why has it gained so much attention? • Ripple effect of “accountability culture” • Influence of business models • Aligns a program’s goals with the institution’s goals • Integrated approach to assessment
Plan … Implement … Assess “If you don’t know where you are going, you will wind up somewhere else.” - Yogi Berra
Elements of Outcomes-Based Evaluation • Documents benefits to program’s participants - What would participants recognize as the benefits? • Focuses on outcomes vs. outputs Output: 51 course-related instruction sessions provided Outcome: Instructors report better use of research materials in students’ papers
Considers changes in: Attitude: What someone feels/thinks about something Skill: What someone can do Knowledge: What someone knows Behavior: How someone acts Status: Someone’s social or professional condition Life condition: Someone’s physical condition From: IMLS, “Outcome Based Evaluation”: http://www.imls.gov/applicants/faqs.shtm
Important Questions to Ask 1. If we had an excellent and successful library user instruction program, what would be the impact or benefits? - administration - faculty - students - library staff - other stakeholders/constituent groups?
2. What would be the characteristics or indicators of success or excellence? - attitudes - skills - knowledge - behavior - status - life condition
3. How might we document this success or excellence? - Examples: interviews observation focus groups performance assessments journals or logs library use statistics
Application to Library User Instruction • Students • Faculty • Library staff • Library services and programs • Administration of the institution
APPLICATION • If you had a successful library instruction program, what would be the impact or benefits? - Describe how the success benefits the people served by the program • What would be the characteristics or indicators of success? - Identify concrete and observable changes in attitudes, skills, knowledge, behaviors, status, and/or conditions • How might you document or demonstrate the success? - What tools or methods would you use to gather data and information?