Research in the Library: An Evidence-based Approach for Making Informed Decisions

1. Research in the Library: An Evidence-based Approach for Making Informed Decisions Library Research Seminar-V Session 3B, Room 1105 Thursday, October 7, 2010

2. The process of implementing a federated search system at the National Institutes of Health Library Anne White-Olson, Information & Education Services Branch Ben Hope, NIHL Information Architecture Branch Douglas Joubert, NIHL Information Architecture Branch DIVISION OF LIBRARY SERVICESOFFICE OF RESEARCH SERVICESNATIONAL INSTITUTES OF HEALTHU.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES

3. Project Goals and Objectives • Find a meta-search product that would allow NIH staff to simultaneously search multiple resources • Establish a search interface that integrates library resources into the NIH Clinical Research Information Systems (CRIS) • Ascertain the needs and preferences of NIH staff in terms of searching for information online • Identify core functionality and ideal user interface design for the NIHL meta-search tool

4. One search system deployed on 2 unique platforms • Clinical Center* • NIH Library The Clinical Center search interfaces exposes patient data; therefore, we are not able to show search results

5. About the search prototype • Developed by Tamas Doszkocs, Senior Computer Scientist at NLM. • Searches multiple, independent databases and clusters retrieval into subject categories. • Utilizes natural language process tools to access heterogeneous information sources on the open and deep web. • Includes spellchecker, automatic mapping to thesauri, concept clusters for focused drill down and query refinement.

6. The Environment - NIH Library 2006 Two things were happening • Professional staff was taking a Qualitative Research Course • Staff was looking into implementing a Federated Search System Outcomes • Combined the two to develop a class project on looking at systems using qualitative research methods learned in class. • In 2009 completing the process implementing First Search at the NIHL

7. Project Phases Phase 1 Phase2 The NIHL contracted with UserWorks, Inc. to conduct a usability study. A qualitative approach using focus groups with six nurse specialists from the Department of Clinical Research Informatics was used.

8. Grounded Theory • Inductively build theories through successive levels of data analysis and conceptual development. • The theory evolves and changes as the researcher interacts with the data. • Methodology is inductive rather than deductive due to the “interplay with the data collected in actual research”. • Interplay is continuous and evolving Berg, B. L. (2007). Qualitative research methods for the social sciences (6th ed.). Boston: Pearson/Allyn & Bacon.

9. CRIS Nurse Informaticians who provide user support, training and documentation for the NIH Clinical Research Information System Our Focus Group

10. Focus Group Methods (1) Participants were asked the following questions What is your favorite web site and why? When you search for information on the web, where do you go? What resources do you use at the library?

11. Focus Group Methods (2) The contents of interviews were transcribed and analyzed in ATLAS.ti according to the principles of grounded theory. • Focus group activities included a discussion of meta-searching and a demonstration of the meta-search prototype. • Additional discussion to elicit feedback from the participants concerning the preferred design and features of a meta-search search system.

12. Focus Group Methods (3) • Focus group transcripts were loaded into ATLAS.ti v5.2.12 for analysis. • Content analysis is the systematic examination and interpretation of a primary document to identify themes and meanings.

13. Focus Group Methods (4) • Independent group also developed codes using index cards. • After the first and second stages of organizing the codes into themes, a third researcher was brought in to resolve differences in organizing the codes into groups.

14. Usability Methods (1) • The usability study compared the prototype to a number of meta-search platforms in an attempt to understand user performance, needs, and preferences of the proposed meta-search platform. • The usability study had 14 participants (4 male and 10 female), which included researchers, administrators, and NIHL staff.

15. Usability Methods (2) • Each subject participated in a one-on-one interview and three user-performed tasks: • Find information on a specific topic • Locate a specific article • If time allowed, conduct a user-defined search

16. Results – Focus Group 88 codes were generated from 114 quotations using the open coding technique, in which research team members grouped each line of discreet and meaningful text from the focus group transcript into conceptual units. • The “Features” theme focused on the manipulation of search results and saving searches • “Types of Resources” theme focused on the resources commonly used by these focus group participants, namely PubMed and CINAHL

17. 16 themes generated from the codes

18. Usability Grouped Categories Advanced search Search terms Search status and results Refining results Clustering Search Results Display Other Key Findings

19. Advanced search • It was important to have an advanced search option to accommodate different users • Labels need to be clearly identified and use intuitive nomenclature (no library jargon) • Help needs be context-specific.

20. Search terms • Increase the prominence of the “did you mean” spell check • Employ commonly used search inputs, such as quotation marks, the word “and,” semi-colons, and the plus sign

21. Search status and results • Wanted a status bar to help them understand when the search was complete. • Preferred to wait and see all the results at once, rather than seeing the results by source.

22. Refining results • Used and liked the refine results option • Function of “refine results” was not consistent across all platforms

23. Search results display • Participants expected and wanted a summary abstract for journal articles • Participants wanted keywords in titles and abstract to be highlighted to determine whether a result was relevant

24. Clustering • Participants thought that clustering was important; however, it needed to be refined: • Offer many layers to the clustering • Show consistent clusters • Show full subcategory cluster headings • Provide a means for narrowing by clustering without losing the primary cluster

25. Other Key Findings • Right-handed content had limited use and interest • Several features were not heavily utilized • Category links • MeSH capability • Publication links • Sort and limit options

26. OK, What Next? • Based on the findings from the usability study, the team developed a list of priorities for the developer • The list was further refined for both products

27. Contact Us • Douglas J. Joubert, MS, MLISNational Institutes of Health LibraryPhone: 301-594-6282E-mail: joubertd@mail.nih.govLinkedIn: http://www.linkedin.com/in/douglasjoubertTwitter: http://twitter.com/doujouDC • Anne White-Olson, MLSNational Institutes of Health LibraryPhone: 301-451-5863E-mail: whiteols@mail.nih.gov