Federated Search : Context, Capability, & Consequence

1. Federated Search: Context, Capability, & Consequence Brenda Reeb, University of Rochester John D’Ignazio, Syracuse University School of Information Studies John Law, ProQuest LITA National Forum 2005, San Jose, California

2. Today • What we did and why we did it – Brenda • How we did it – John D’Ignazio • What we found – John Law • Capability and Consequences – John Law • Questions – Audience! Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

3. Reasons why • Animal, mineral, or vegetable? • Bridge from federated search results to native resource – or not? • Find information vs. analyze information – transparent technology and resources Context Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

4. How we did it

5. Inserting a new technology into a complex system Questions multiply! • How does it work with mass of database offerings? • How does it relate to the existing web implementation? • How do you value the additional system responses? • How does it relate to Google? How to answer? • Keep building… • Or, Take it to the user? Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

6. Inspiration: A Formative User Study • comfort with usability testing to evaluate library web design • contextual inquiry: put tech development in context of user’s information tasks • compare user work practice in library’s web environment • w/ federated search • without Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

7. Elements of Contextual Inquiry An interview technique: • Master-apprentice model BUT user is master! • Think aloud protocol…or in our case Think After “Tell me about the paper you last wrote.” User then dives away from description toward concrete actions taken to reach goal • Establish easy manner to elicit practices and user relationships with technology Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

8. University of Rochester Brenda Reeb -- usability program coordinator Nancy Foster -- anthropologist/researcher Nora Dimmock -- multimedia center ProQuest John Law -- director, product management Endeavor Mike Visser -- product manager Association for Research Libraries - ARL John D’Ignazio -- usability consultant Subjects - 9 3 first year 6 upper years 3 social sciences 3 humanities 2 sciences 1 undeclared Solicited from library usability email roster Did you write a paper last semester? Received $25 gift certificate Participants Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

9. Test Setup Moderator/Apprentice • sits next to subject • guides, prompts subject through task according to script • What are you doing now?,Why did you do X instead of Y?, What would you like to do right now? Show me. • Have you ever used federated search…let’s try. Note-taker • records interaction, • concentrates on user attitudes/approach Subject and subject’s actions simultaneously recorded Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

10. Morae Software Suite • Event recording software, made by TechSmith • Flexible, easy, low cost • 3 modules: • Morae recorder: capture audio, video and user actions • Morae remote viewer: observe a test from a remote location • Morae manager: analyze the results and create visual presentations Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

11. Room for Improvement • Test Subjects: • not chosen randomly; one third were library employees • Moderators: • hard to play apprentice while guiding progress • Test Setup: • not native work environment • recreating past actions may lead to idealization or omissions • forced switchover to federated search • Fitness of Method • adopted from HW/SW development community; libraries may need different formative/evaluative methods Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

12. What we found • Observations • New capabilities for federated search • Consequences • Conclusions Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

13. Observations • Students’ search strategies are well considered (albeit more literal than informed) • Users value their time highly and are unwilling to “fish around” long for appropriate resources Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

14. Video snippet #1 Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

15. Observations (con’t) • Everybody wants a summary/abstract • Give me enough information to decide which items are worth pursuing • Where are my search terms in my results? • Nobody likes complex paths to full text • Users expect that search results are presented in order of most relevant Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

16. Video snippet #2 Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

17. Observations (con’t) • “Search all” usually not appealing in searches of library resources • Little known distinction between different databases • Frequently accessed target e-resources via course pages • Context adds to value perception: “Faculty-recommended resources” Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

18. Video snippet #3 Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

19. Even more observations • Trusted resources, whether appropriate for the research project or not, win over a “search all” federated view of library resources • Resources discovered “in context” are considered as trusted and credible (otherwise, little perceived distinction between resources) • Fluid interaction among tools and resources (e.g. Google, Amazon, NoodleBib, etc.) Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

20. New capabilities for Federated Search • Provide evaluative information in search results. Why pick this one? • abstract/summary, subject term highlight, etc. • Provide direct access to full text • no confusing intermediate pages (e.g. link resolver page, citation page, etc. • predictors of full text availability (i.e. don’t show a full text link unless full text is known to be available Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

21. New capabilities (con’t) • Conundrum: • Based on current technology, “Search all” federated searching of premium scholarly resources falls short of satisfying user needs • Web search engines’ superiority in relevancy sorting of search results is found more satisfactory although lacking in scholarly integrity Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

22. Consequences • Students return to past trusted resources. Is this wrong? • This is compensatory behavior: • When resources are not available in the context of their work • When it’s unclear where to go for quick, reliable answers to research questions • Student focus is course-centric • The “home page” is the course page • Learning management systems and student personalized portals will become “information central” as these solutions evolve • Library resources need to be available in these environments • Selective content presented in context Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

23. Conclusions • Context adds value and trust • Resources associated with a course were perceived as superior • Clearly, students are engaged in their research • an appropriate combination of resources (and tools) need to be available in-context • Users differentiate between authoritative scholarly resources and general web search results, however: • the latter is easier to deal with since it provides a predictable quality of search results, while • the former is inconsistently presented and difficult to discover in a manner consistent with user needs, expectations and current practices Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005

24. From the user’s perspective: Finch Hormones Lab Report December 2004 How do finch hormones impact birdsong? Federated Search – Reeb/D’Ignazio/Law LITA Forum 2005