Supporting Cyberscholarship in American Social History: The DLF Aquifer Story

1. Katherine Kott With content contributed by Susan Harum and Chick Markley CNI Task Force Meeting December 11, 2007 Supporting Cyberscholarship in American Social History: The DLF Aquifer Story

2. Focus on American Social History Online • Mellon funded project within DLF Aquifer initiative • Outcome to make digital collections in American culture and life easier for scholars to find and use • Aggregate content to enable multiple collections to be used as one • Deliver content through multiple channels

3. Who’s who • American Social History Online relies on Aquifer working groups, collaborative infrastructure in place • Core team focused on development and assessment, coordination with working groups • Susan Harum (UIUC) business analyst/assessment expert • Kat Hagedorn (Michigan) data analyst • Chick Markley (Freewheeling contractor) systems architect/developer • Tom Habing (UIUC) developer

4. Bibliotheca Alexandrina California Digital Library Cornell University Emory University Indiana University Johns Hopkins University Library of Congress New York University Stanford University University of Illinois, Urbana-Champaign University of Michigan University of Minnesota University of Southern California University of Tennessee University of Virginia Yale University DLF Aquifer participant libraries

5. Overview of accomplishments • Architecture • Portal with Zotero—initial SEO analysis • MODS • Guidelines • Levels of adoption • MARC to MODS transform • Harvesting workflow • Agile development process

6. Website with Zotero in Firefox RESERVED

7. How we got here • Initially unsure we would build a website • SEO required for commercial search service scenario • Zotero implementation requires website • Website seemed like best way to present services (SRU) for Sakai and federated search scenarios • Website could be used as cohesive way to present the project • In June 2007, team set goal to have website ready for DLF Fall Forum in November 2007

8. Website development • Started with baseline functionality on Michigan MODS portal • Added and cross checked common business functions developed by SWG (integrated with DLF Services Framework) • Architecture chosen to support • Website functionality • Sakai • Federated search

9. TWG architectural framework

10. Architectural implementation ToJSON java to JSON java Harvester DLXS Rails ruby JSON mysql services MODS mysql SOLR/Lucene java to SOLR java bots browsers eXist java to eXist java

11. A plug for Ruby on Rails from our systems architect • It's the framework • DRY (don't repeat yourself) • Tightly coupled testing harness • Exuberant and dedicated community

12. Enter, Agile development Small development team, distributed environment Short timeline—no time for “traditional” FRD process Desire within digital library community for lightweight process model Emphasis on user driven development Not to be confused with final assessment activities for American Social History Online

13. Model Timeframe: August 29 – November 2, 2007 (8 weeks) Business Analyst identified 8 participants 1 faculty member 3 graduate students 4 user services librarians Services Working Group and Business Analyst drafted questions

14. Method • Initial conference call on August 27th, 2007 • all participants • business analyst • systems architect • developer • Questions regarding • search and find • identify • obtain • manage and use • user workflow

15. Initial Brainstorming call Pros developers present to hear what participants say developers able to prioritize next tasks immediately after the call Cons Level of participant feedback was lower than anticipated Too much developer involvement?

16. Participant feedback and development action Need to see size and scale of collection to determine how much time to spend searching and browsing Implementation of collection registry and “browse collections” feature. Need ability to limit a search (once you have results) in many ways Implementation of additional ways to limit after initial search Implementation of date range limiting Importance of ability to browse Implementation of subject clouds for browsing – still working on faceting for additional browsing Importance of visual aids, especially for undergraduate students Implementation of Simile timeline view for results Thumbnail previews for item selection

17. Integration with collection registry at UIUC

18. Adaptively re-used technology • CDL date normalization utility • UIUC thumbgrabber with asset actions • Simile timeline from MIT • Collectus tool from UVA • Just say no to NIH syndrome

19. Implementation of “asset actions” using Collectus from UVa

20. Simile Timeline Integration

22. Participant wish lists for future development Limiting Search Results users want to “drill down” in the search continue to narrow using different limiters, e.g. rights for use Social Networking and Tagging Implementation of StumbleIt for full records Monitoring Zotero server development Penn tags Sorting and Narrowing Plan to implement faceted browsing based on NINES Collex flare development (NIH)

24. Tell me something I don’t know

25. Lessons learned Agile Assessment effective in prioritizing tasks Multiple phone interviews and email updates resulted in buy in for participants Brainstorming session for distributed group not as effective as personal interviews Shorter more frequent 1:1 interviews yielded the best results Graduate students were most engaged

26. Add collections Workflow, including asset actions Support for static repositories Bug fixing Tagging? Faceted browse Map rights statement to CC license? Evaluate ORE model (March meeting) Develop “local implementations” Sakai @ Indiana Federated search @ UIUC More SEO Assessment planning Sustainability planning What’s next

27. Actively seeking collections • Advantage to contributor • SEO drives traffic to your collection(s) • Benefit from date normalization • Seeking collections • Pertaining to American culture and life • All formats • MODS—can be mapped from MARC • Static repository setup for contributors without OAI data providers

28. The DLF Aquifer engine • From DLF for development to ??? for sustaining • Consistent attention • Administrative support • The working groups • Organizational links • Community links • Resources • From collaborators (directly and through investment in DLF) • Grant funding • Fees???

29. Sustaining Aquifer • Part it out • Community standards • Software and services • Collections • Take it to the next level? • Affiliate with another (service provider) organization • Move to participant library host

30. Resources for ongoing support “If enough people who are viewed as important want it, we will find a way to pay.” Paul Courant • Assessment during development period is critical • Revenue streams • Subscription • Libraries pay for services • Increased visibility for collections • Metadata enhancement

31. Emerging organizations sustaining inititiatives • LOCKSS/CLOCKSS • Fedora Commons • DSpace Foundation • Educopia Institute

32. Website: www.dlfaquifer.org kkott@diglib.org Questions? Comments?