Digital Libraries with Greenstone: an open source solution

1. Digital Libraries with Greenstone:an open source solution Tod Olson - University of Chicago Fred Miller - Illinois Wesleyan University Curtis Kelch - Illinois Wesleyan University

2. Digital Libraries with Greenstone • Introduction • About digital libraries • Greenstone overview • Examples • Future • Live demos • Q & A

3. The World of Digital Libraries • Access to Digital Collections • Text, images, audio, video • Searching and metadata • Digital libraries versus repositories • Access and preservation • Digital Preservation Tutorial http://www.library.cornell.edu/iris/tutorial/dpm/

4. Sorting Out the Ingredients • Raw materials • User interface • Elements of organization • Building the collection

5. Greenstone New Zealand Digital Library Projectat the University of Waikato • with UNESCO, Human Info NGO International, every continent Examples: • Academic • Digitization projects • Classes on digital libraries • Non-academic • UNESCO humanitarian documentation

6. Greenstone features • Works with existing documents • Imports several formats • Searching: full text and metadata • Dublin Core, custom metadata • Browse • Structured documents • Indexing, access • Extensible & customizable • OpenSource software (GPL)

16. Greenstone Architecture Receptionist Receptionist Protocol Collection Server Collection Server Collection Collection Collection DB & Indexes DB & Indexes DB & Indexes Import Import Import Redrawn from Witten & Bainbridge, How to Build a Digital Library, p. 356

17. Receptionist Provides user interface Accept user input Send to appropriate collection server Accept results Dynamic page generation Collection Server Handle collection content Search and filter information Return results multiple collections Greenstone Architecture

18. Building Collections HTML DB & Indexes Import Build PDF GSAF ???

19. Building collections • Create a collection framework • or work with an old collection • Select documents • Import documents • Converts to internal XML format (GSAF) • Build collection • creates search indexes and browse listings

20. GSAF: internal XML format Section: • Description • Metadata fields • Content • Text,internal markup, images • Section • No limit in number or depth Hierarchical documents Sections nest, tree structure

21. GSAF: internal XML format <Section> <Description> <Metadata name=“Title” value=“…”> <Content> [Text, images, links, etc.] <Section> <Description> <Metadata name=“Title” …> <Content>… <Section>… <Section>… <Section>…

22. Config file: collect.cfg Collection-specific configuration file, collect.cfg, specifies:   • file types to import • Indexes and browse lists • Document or section level • paragraph (text index only) • display of results and browse listings • document displays

28. Chopin Early Editions Over 400 early edition Chopin scores 1830’s to 1880’s Target audience: music scholars & musicians. On web, page-turnable JPEG images. Online in March 2003 Currently 374 scores in online collection Usage: Nearly100 hits per day, > 30% of use is international.

36. Catalogrecords ScannedImages Structuralmetadata Build overview Human processing GreenstoneArchiveFormat GreenstoneDig. Library Software XSLT METS XML-based automated processing

37. METS to GSAF Section Description Metadata: Title, … Content: Title, … Section Content: Page 1 page1.jpg Section Content: Page 2 page2.jpg dmdSec MODS: Title, … fileSec page1.jpg page2.jpg structMap div: Score div: Page 1 div: Page 2

38. Greenstone benefits for Chopin • Robust, mature system • Recovered time in project • Fast to bring up • UI out of the box • Dynamic page generation • Incremental customization • XML compliant • Natural mapping from METS to GSAF

39. The Argus Digital Collection • Illinois Wesleyan Student Newspaper • 1894 to 2000 • Preservation and Access • Image PDF versus full text • Web interface for building metadata • Customized searches

40. Argus Metadata Maintenance

41. Argus Search

42. Argus Issue “front door”

43. Ongoing work: Greenstone • Greenstone Librarian Interface (GLI) • Greenstone 3

44. Collection management Informed by work at GS sites Assist collection designer Support all phases of collection build process Do not specify workflow Java-based GUI tool Formerly called the “Gatherer” 2 yrs in development Beta sites: Bangalore and elsewhere Training sessions UNESCO sessions in Asia, Africa JCDL 2004 tutorial Greenstone Librarian Interface (GLI)

45. Greenstone 3 GS2 mature, 5+ yrs., wide deployment • Constraints: support legacy systems • Other technologies have matured: Java, XML GS3: rewrite in Java, XML, XSLT • Distributed architecture, SOAP • METS as internal format • Group assembled for Greenstone METS profile(s) • OAI support planned • 1 year in dev; alpha testing in lab

46. Links & Further Information Greenstone: http://www.greenstone.org/ Chopin Early Editions: http://chopin.lib.uchicago.edu/ Argus Digital Collection: http://www.iwu.edu/library/services/argus1.htm Argus Greenstone Documentation: http://www.iwu.edu/~ckelch/ArgusProjectDoc12.pdf Witten & Bainbridge. How to Build a Digital Library. Morgan Kaufman, 2003.

47. More about Greenstone…