Measuring Your Impact: Using Evaluation for Library Advocacy

1. Measuring Your Impact: Using Evaluation for Library Advocacy GAIN SPRING MEETING, MERCER UNIVERSITY, MACON, GA April 24, 2014

2. Instructor Andrew Youngkin NN/LM SEA Evaluation and Technology Coordinator ayoungki@hshsl.umaryland.edu

3. Agenda

4. Workshop objectives By the end of the workshop you will: • Understand your library’s value in terms of the mission of the larger organization • Describe some tools used to assess your library, its users and stakeholders • Identify the tools and methods used for data collection and analysis • Design an evaluation plan for a service in your library • Communicate evaluation results

5. What this workshop is not… • Custom-tailored • A quick fix • One size fits all

6. “What gets measured, gets done.“ Proving to your administration that money spent in your library is going to pay student achievement dividends is well worth the time and effort http://www.doug-johnson.com/dougwri/what-gets-measured-gets-done.html

7. Demonstrating YOUR library’s value

8. Why demonstrate value? • To show the impact of your library on the larger organization’s mission and goals • To show accountability for your resources; your library’s contribution to the bottom line • To demonstrate that your library provides value-added services • To use as an advocacy and marketing tool

9. How & what to evaluate • What you decide to evaluate depends on: • What you need or want to know • What your users feel is important • What certain stakeholders want to have evaluated • Use “smallball” evaluation • Base hits or home runs?

10. Library service is value-added • Cost savings • Institutional & Personal • Time savings • Travel, discovery, skills acquisition, opportunity for other uses of time • Access • Opportunity to use/apply information • Impact on learning, research

11. The bottom line • You are either generating revenue, or supporting those who do OR … • You are helping to control operating expenses or supporting those who do OR … • You are creating expenses that add recognized value OR … • You are creating expenses that must be controlled or eliminated to reduce overhead Will Welton, PHD, Director, MHA Program, University of Washington

12. The big question! What is the bottom line and what is your library doing to support it?

13. What you need is a plan • Know the vision • Look at the environment • Create the plan • Collect and analyze the data • Communicate the value

14. The vision

15. Know what you want to do and why • Align your library’s mission & goals with the organization’s mission & goals • Your library’s value = library’s contribution to achieving organization’s mission & goals

16. Mission statements • dedicated to cultivating the inherent gifts, skills and talents of children in Grades 6 through 12 so that they are prepared for success in college and life. • to provide resources that support the curriculum, help our students to develop habits to become lifelong learners and self-actualized citizens, and promote understanding and appreciation of our Marianist heritage.

17. Mission statements • ...foster in our students academic, physical and creative fulfillment, together with strength of character, while helping them become productive members of our school community. • …support and … foster the active and creative pursuit of academic and artistic achievement through the effective use of information and the encouragement of curiosity and scholarly inquiry.

18. HAIKU creative fulfillment,academic and physical;productive community ...foster in our students academic, physical and creative fulfillment, together with strength of character, while helping them become productive members of our school community effective information useencourages curiosity andscholarly achievement …support and…foster the active and creative pursuit of academic and artistic achievement through the effective use of information and the encouragement of curiosity and scholarly inquiry

19. Write a haiku • Write your mission statement or another message • Turn that into a haiku • Standard haiku is 5/7/5 syllables, but English haikus are more flexible because of our syllable structure • Share your haiku

20. Made to stick • SUCCESs • Simple – core, compact • Unexpected – gets attention • Concrete – specific, not abstract • Credible – testable • Emotional – more than self interest • Stories – stimulate, inspire (From: Made to Stick. Heath, Chip. New York: Random House, 2007)

21. The environment

22. Why look at the environment? • Understand needs, desires and problems in context • Validate assumptions about your contributions and services • Provide a baseline for future evaluation • Help to develop the blueprint to plan and evaluate your contributions and services

23. Components of environment • The organization • Your library • Clients/users • Stakeholders • The community

24. Assessing the environment • User/stakeholder input • Surveys, focus groups, interviews • Unsolicited feedback • Conversations • Observation • Assessment by walking around • Know the language of your users • Information/literature • Journal/serial/magazine

25. Assessing the environment • Library statistics and records • Collection use • Librarians’ work including reference, teaching, etc. • Get out of your library and talk to people • Understand the role of information in their work • Learn how information is acquired, applied • Discover information problems they are trying to solve • SWOT analysis • Understanding strengths, weaknesses, opportunities, threats • Consider how these affect your work, your assessment

26. Librarians’ top 10 tasks • School Librarians say… • Help students select books • Collaborate with teachers • Read and book talk with classes • Teach research and use of technology to students and teachers • Keep library organized including cataloging, weeding, inventory, late notices, vacuuming, dusting • Study standards to prepare library lessons • Troubleshoot technology issues around the building • Help with extracurricular activities including laminating, school plays, lunch duty, etc • Prepare book, equipment and supply orders • Organize special library events like book fairs, author visits, contests

27. Librarians’ top 10 tasks • School Principals say… • Help students access information and books • Help faculty access information and books • Share technology expertise with students and teachers • Select “appropriate” materials • Model love for reading • Collaborate with teachers • Provide equipment and technology • Provide leadership with technology • Teach research skills, teach about books and databases • Provide an inviting environment How does your boss see you? Proof that principals value librarians. School Library Journal http://www.slj.com/2012/09/careers/how-does-your-boss-see-you-proof-that-principals-value-librarians/#_

28. Librarians say Principles say

29. SWOT analysis Strengths • Internal • Positive statements about your library Weaknesses • Internal • Statements about what is lacking in your library Opportunities • External • What do stakeholders want you to do that you are not doing? Threats • External • Factors that can adversely impact your library’s goals

30. Plan Backward, Implement Forward

31. The logic model - a tool to… • Help organize your thoughts – both before and during a project/program • Provide a framework for planning and evaluating programs • Clarify intended outcomes • Acts as a communication tool Good source for more information: W.K. Kellogg Foundation Logic Model Development Guide (pdf on your thumb drive; see class bibliography for link)

32. Pieces of the logic model • Outcomes • Activities • Resource • Outputs

33. Outcomes • Outcomes - changes in attitude, behavior, skills, knowledge or situation • Short, medium and/or long term • Intended or unintended • Positive as well as negative • S.M.A.R.T.

34. SMART Outcomes Outcome: Health resources Web pages are available on the library website by the end of the project • Specific: one or more web pages with links to EBM resources are available on the library website • Measurable:done or not; do nurses access resources • Action-oriented: created and posted; nurses access resources • Realistic: It is possible within the timeframe • Timed: “when” is articulated

35. OUTCOMES Outcome • Long Term: • Patients and their families will have improved health information literacy • Intermediate Term: • Staff will have information for timely clinical decision-making • Short Term: • Health resources Web pages are available on the library website by the end of the project

36. Activities and resources • Activities • What will you do? • Resources • What you have • Income • Equipment • Collection • What you need • Operating expenses (e.g., personnel, acquisitions, maintenance, etc.) • Funds for new initiatives or services

37. Outputs • Outputs • How many did you do? • How many attended? • How many were distributed? • How many times was it used?

38. Putting it all together Resources If we get these resources… Activities/Outputs Conduct these activities and deliver these products Outcomes We will see these outcomes

39. The model may change over time Resources Activities Outcomes Outputs

40. Logic model

41. Logic model • Outcomes: ST: Health resources Web pages are available on the library website by the end of the project; IT: Staff will have information for timely clinical decision-making

42. Logic model group activity • Identify outcome(s) • Articulate short/intermediate/long term • List • activities • outputs • resources

43. The evaluation plan

44. The evaluation plan • Builds on the logic model • Evaluates success • Assesses value

45. Consider the purpose • Who is your audience? This may be different than the beneficiaries of your services • Your users • Administrators • Patients/families • How will the information be used? • Financial savings or justification • Intangible or non-monetary value of program benefits to community • Marketing and advocating for your library

46. What do you want to know? Have you achieved your stated outcome? • Short Term: • Health resources Web pages are available on the library website by the end of the project • Intermediate Term: • Staff will have information for timely clinical decision-making • Long Term: • Patients and their families will have improved health information literacy

47. When do you want to know? • Consider objective(s) • Articulate the time frame

48. Evaluation plan • Outcome (from logic model):Staff will have information for timely clinical decision-making

49. Example part 1 • Outcome (from logic model):Staff will have information for timely clinical decision-making

50. Example part 2