Assessing Our Value: This is Our Story

1. Assessing Our Value: This is Our Story Alicia Livinski, Diane Cooper, Bradley Otterson, Nancy Terry, Terrie Wheeler, Anne White-Olson National Institutes of Health (NIH) Library

2. National Institutes of Health (NIH) Library • Federal library supporting a biomedical research institution • NIH employees, students, fellows, visiting researchers etc. • Also serve various US Department of Health & Human Services (HHS) staff and operating divisions • HHS users include: policy analysts, budget analysts, healthcare providers, mental health providers, administrative staff, program officers etc.

3. HHS users include: • Administration for Children & Families (ACF) • Administration for Community Living (ACL) • Agency for Healthcare Research & Quality (AHRQ) • Assistant Secretary for Financial Resources (ASFR) • Assistant Secretary for Planning & Evaluation (ASPE) • Assistant Sec for Preparedness & Response (ASPR) • Health Resources & Services Administration (HRSA) • Indian Health Service (IHS) • Office of the Asst Secretary for Health (OASH) (+ 14 sub-offices) • Office of General Counsel (OGC) • Office of Global Affairs (OGA) • Office of the Inspector General (OIG) • Office of the National Coordinator for Health IT (ONC) • Office of Civil Rights (OCR) • Program Support Center, Federal Occupational Health • Substance Abuse and Mental Health Services Administration (SAMHSA)

4. Purpose of the study • Specifically, this assessment will seek to address whether the services and information provided by National Institutes of Health (NIH) librarians, as perceived by our HHS users, impacted our user’s decisions and/or actions related to patient care, policy, administrative, research or educational work.

5. Study design • Setting: federal biomedical research library • Assumptions:Individual survey responses will be confidential, and participation is entirely voluntary. • Population: Health policy, public health professionals, clinicians and other healthcare professionals from HHS. Our population (N) are the HHS individuals served by the NIH librarians. The survey will be administered to a convenience sample (n) of the people served beginning in April 2012 to when 20% of all total incidents in 2011 (162) is reached. In 2011, the total incidents for HHS customers completed by the NIH librarians were 811.

6. Study design • Questions: were primarily based on the survey questions developed and used for the Federal Libraries Value Project in 2010 & 2011; • Patient care questions from previous studies of federal medical libraries; • Policy questions were developed with input from HHS users & review of public health policy and knowledge management literature;

7. Study design • Methods: critical incident technique was used • SurveyMonkey™ • Study questions were piloted prior to launching the survey • Began in April 2012 and will end when at least 162 surveys are completed • Librarians email a link to the survey after completing the following: document delivery by the librarian; literature searches; reference questions; editing/formatting of manuscripts; instruction on searching • Each librarian enters the # of distributed surveys into a Excel spreadsheet

8. Survey results to date • As of April 2013, 73 surveys completed (out of 162 needed); Q: Job titles of respondents: • Physician (10%) • Program analyst (7%) • Fellows (6%) • Nurse (6%) • Public health analyst (6%) • Policy analyst (6%) • Executive staff (4%) • Other (31%)

9. Question asked of all users

10. Purpose of information request (n=69)

11. Asked of all users Q: Thinking about the information you received, how much influence did it have on your work?

12. Asked of all users Q: If you had not asked the NIH librarian, how would you have sought the information? • Searched Google = 29% • I would not have searched = 18% • Asked a colleague = 10% • Other = 43% • I would have been unable to meet my goals; • Asked subordinates/colleagues/contract staff; • Search databases, PubMed, Google, journals myself; • Bought the book for a lot of money; • I would have tried my university alumni library

13. Patient care

14. Patient care (n=5) 100% of respondents indicated that the information provided by the NIH Librarian was useful for direct patient care.

15. Patient care (n=5) Q: After reading the information provided, how did it influence you? • Provided me with standard of care info = 60% • Advice made to patient or family = 40% • Reinforced my plan of treatment = 40% • Changed the length of stay = 20% • Improved patient management = 20%

16. Patient care (n=5) Q: Were any of the following events avoided because of the information provided? • I don’t know = 40% • Additional test or procedures = 20% • No events were avoided = 20% • Other: = 20% • program activities or plans based on outdated info can be avoided because of this information

17. Patient care (n=5) Q: Did the information provided have a potential financial impact for either patient or the medical center. • I don’t know = 40% • Avoided unnecessary costs = 20% • Other: = 40% • Potential for cost savings based on prevention of adverse health outcomes; • May increase pharmacy expenses while decreasing morbidity & mortality;

18. research

19. Research (n=36) Q: What research functions did the information you received assist you with doing? • Publication in peer-reviewed literature = 36% • Technical/scientific report = 33% • Decision making = 33% • Practice guidelines = 31% • Protocol development = 31% • Systematic review = 25% • Treatment improvement guidelines = 19% • HHS publication/report = 14% • Consensus development = 8% • Conference presentation = 8% • Other: = 14%

20. Research (n=36) Q: Was the research part of obtaining research funding or IRB/IACUC approval?

21. Research (n=4 who answered yes to previous question) Q: Please estimate funding to be sought or received. • 1 million – 1.9 million = 25% • 100,000 – 249,000 = 25% • Not applicable = 50% Q: Select statement that best represents the financial impact of the information provided on your research. • Avoided unnecessary costs = 50% • Support for a research proposal = 25% • Other: = 25% • Avoidance of repeating studies due to technical errors

22. Policy or administrative

23. Policy or administrative (n=18) Q: Which best describes the outcome(s) of the information requested? • Review agency policies/procedures = 39% • Conduct program evaluation/development = 33% • Conduct policy review/implementation = 33% • Identify best practices = 28% • Perform legal research = 17% • Identify collaborators/business opportunities = 11% • Conduct research impact analysis = 11% • Other = 17% • Review of social phenomenon to guide policy research; online news monitoring

24. Policy or administrative (n=18) Q: How would you describe the value of the information received from the NIH Librarian? • Helped me to make a more informed decision = 67% or recommendation • Confirmed my decision or recommendation = 39% • Increased my confidence in making a decision = 22% or recommendation • Refreshed my memory of details or facts = 11% • Caused me to consider another approach = 6% • Other = 22%

25. education

26. Education (n=10) Q: Did the information you requested support one or more of the following? • Your own professional development = 50% • A class you are taking = 20% • Background information for speech/paper = 20% • Other = 50% • Health systems redesign; • Improve nutrition counseling we provide; • Grant writing efforts; • Information for tribal health partners

27. Challenges • Remembering to include the survey when responding to information requests (and sending follow up) • Question development was a challenge – • a majority of our users are not healthcare providers or bench scientists (although the Indian Health Service does provide most of the clinical care) & • how do we measure our impact on policy, analysis or other public health focused work

28. Conclusion • We are not done yet. We had anticipated completing our data collection and analysis by May 2013; however, we are still collecting our data. • However, these preliminary results are positive and are beginning to show us how our HHS users perceive the impact of our services on their work. • We ask that anyone working with similar user populations and are interested in question development to please contact us.