1 / 56

AOL Search Speaker Series Virginia Tech’s Digital Library Research Laboratory

Explore DL integration problem formalization, modeling, mapping, and architecture in the context of a UnionDL. Assess integration quality, interoperability, and design. Example of Union Services provided by CitiViz in the CITIDEL system. Acknowledgements to key contributors in the research.

ashumaker
Download Presentation

AOL Search Speaker Series Virginia Tech’s Digital Library Research Laboratory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AOL Search Speaker SeriesVirginia Tech’s Digital LibraryResearch Laboratory Dec. 20, 2004 -- AOL HQ Edward A. Fox, fox@vt.edu Virginia Tech, Blacksburg, VA 24061 USA http://fox.cs.vt.edu/talks/2004/ http://fox.cs.vt.edu/cv.htm

  2. Acknowledgements (Selected) • Sponsors: ACM, Adobe, AOL, CAPES, CNI, CONACyT, DFG, IBM, Microsoft, NASA, NDLTD, NLM, NSF (IIS-9986089, 0086227, 0080748, 0325579; ITR-0325579; DUE-0121679, 0136690, 0121741, 0333601), OCLC, SOLINET, SUN, SURA, UNESCO, US Dept. Ed. (FIPSE), VTLS

  3. Acknowledgements: Faculty, Staff • Lillian Cassel, Debra Dudley, Roger Ehrich, Joanne Eustis, Weiguo Fan, James Flanagan, C. Lee Giles, Eberhard Hilf, John Impagliazzo, Filip Jagodzinski, Rohit Kelapure, Neill Kipp, Douglas Knight, Deborah Knox, Aaron Krowne, Alberto Laender, Gail McMillan, Claudia Medeiros, Manuel Perez, Naren Ramakrishnan, Layne Watson, …

  4. Acknowledgements: Students • Pavel Calado, Yuxin Chen, Fernando Das Neves, Shahrooz Feizabadi, Robert France, Marcos Goncalves, Nithiwat Kampanya, S.H. Kim, Aaron Krowne, Bing Liu, Ming Luo, Paul Mather, Saverio Perugini, Unni. Ravindranathan, Ryan Richardson, Rao Shen, Ohm Sornil, Hussein Suleman, Ricardo Torres, Wensi Xi, Xiaoyan Yu, Baoping Zhang, Qinwei Zhu, …

  5. Rao Shen’s Preliminary Exam:Hypothesis and Research Questions • The 5S framework provides effective solutions to DL integration. • Formally define the DL integration problem? • Guide integration of domain focused DLs? • How to formally model such domain specific DLs? • How to integrate formally defined DL models into a union DL model? • How to use the union DL model to help design and implement high quality integrated DLs? • Assess the integration?

  6. Consists of mediator wrapper agent Intermediary-based mapping-based Interrelated with use hybrid mapper use composite mapper schema mapping used in use federation Union Archiving two architectures Consists of has an example has an example SemInt LSD Related Work DL interoperability approach

  7. Consists of mediator wrapper agent Intermediary-based mapping-based Interrelated with use hybrid mapper composite mapper schema mapping used in use federation Union Archiving two architectures Consists of DL integration formalization based on DL interoperability approach use trained by GA

  8. Formal Definition of DL Integration • DLi=(Ri, DMi, Servi, Soci), 1 i n • Ri is a network accessible repository • DMi is a set of metadata catalogs for all collections • Servi is a set of services • Soci is a society • UnionRep • UnionCat • UnionServices • UnionSociety

  9. Formal Definition of DL Integration (Cont.) • DL integration problem definition: Given n individual libraries, integrate the n DLs to create a UnionDL. Demonstration: ETANA-DL (NSF ITR w. CWRU) feathers.dlib.vt.edu

  10.     Society Society Union Society     General Public archaeologists Archaeologists General Public Architecture of a Union DL DL1 Union DL DL2 Union Service Service Service Harvesting, Mapping, Searching, Browsing, Clustering, Visualization Searching Browsing Union Catalog Catalog1 Catalog2 Union Repository Repository1 Repository2

  11. Mapping Tool Union Catalog Wrapper Wrapper Mapping Tool Union Catalog Integration Virtual Nimrin (VN) VN Metadata Format Union ArchDL VN Catalog Global Metadata Format Halif DigMaster (HD) HD Catalog HD Metadata Format

  12. Example of Union Service: CitiViz

  13. CitiViz:A Visual User Interface to the CITIDEL System ECDL 2004, Bath, England, September 2004 Nithiwat Kampanya, Rao Shen, Seonho Kim, Chris North, and Edward A. Fox fox@vt.edu http://fox.cs.vt.edu

  14. Structures Societies Scenarios hypertext Streams indexing Spaces searching services Collection Repository browsing A Minimal DL in the 5S Framework Structured Stream Structural Metadata Specification Descriptive Metadata Specification Metadata Catalog Digital Object Minimal DL

  15. Streams ArchObj ArchColl StraDia SpaTemOrg ArchDR hypertext services ArchDColl browsing indexing Societies Scenarios Spaces Structures searching Descriptive Metadata specification Structured Stream Arch Descriptive Metadata specification Arch Metadata catalog ArchDO Minimal ArchDL A Minimal ArchDL in the 5S Framework

  16. ArchDL Expert 5S Archaeology MetaModel ArchDL Designer 5SGraph Structure Sub-model VN Metadata Format HD Metadata Format Scenario Sub-model ETANA-DL Metadata Format VN Catalog HD Catalog Mapping Tool Wrapper4VN Wrapper4HD Component Pool 5SGen Browsing … ETANA-DL Union Services Descriptions Harvesting Mapping Searching Browsing … Inverted Files XOAI Web Interface Search Service Index Union Catalog Browse DB Index Browse Service Services DB Other ETANA-DL Services XOAI

  17. Computing and Information Technology Interactive Digital Educational Library (CITIDEL) • Domain: computing / information technology • Genre: one-stop-shopping for teachers & learners: courseware (CSTC, JERIC), leading DLs (ACM, IEEE-CS, DB&LP, CiteSeer), PlanetMath.org, NCSTRL (technical reports), … • Submission & Collection: sub/partner collections  www.citidel.org

  18. www.CITIDEL.org • Led by Virginia Tech, with co-PIs: • Fox (director, DL systems) • Lee (history) • Perez (user interface, Spanish support) • Students: Ryan Richardson, Kate McDevitt, Jon Pryor, Baoping Zhang • Partners • College of New Jersey (Knox) • Hofstra (Impagliazzo) • Villanova (Cassel) • Penn State (Giles)

  19. Digital library architecture for local and interoperable CITIDEL services

  20. CITIDEL Technology Features • Component architecture (Open Digital Library) • Re-use and compose re-deployable digital library components. • Built Using Open Standards & Technologies • OAI: Used to collect DL Resources and DL Interoperability • XSL and XML: Interface rendering with multi-lingual community based translation of screens and content (Spanish, …) • Perl: Component Integration • ESSEX: Search Engine Functionality • Very fast, utilizing in-memory processing • Includes snap-shots for persistence • Multi-scheming (Aaron Krowne, now at Emory U. Library) • Integrates multiple classifications / views through maps, closure • Extensions: clustering, visualization, personalization, …

  21. Cluster Search Results from CITIDEL

  22. Cluster NDLTD-Computing

  23. Naren Ramakrishnan and Saverio Perugini (U. Dayton) CITIDEL + PIPE • Adds Interaction Personalization to CITIDEL • Automatically handles multi-modal conversion to Cell phone, PDA, Etc. • Can be adopted to any digital data set, only requires XML file of content with hierarchy maintained.

  24. CITIDEL -> NSDL • A collection project in the • National STEM (science, technolgy, engineering, and mathematics) education Digital Library – NSDL • National Science Digital Library • www.nsdl.org • (Next slides courtesy Lee Zia, NSF)

  25. NSDL ProgramTracks • Core Integration:coordinate a distributed alliance of resource collection and service providers; and ensure reliable and extensible access to and usability of the resulting network of learning environments and resources • Collections:aggregate and actively manage a subset of the digital library’s content within a coherent theme / specialty • Services:increase the impact, reach, efficiency, and value of the digital library in its fully operational form • Targeted (Applied) Research:have immediate impact on one or more of the other three tracks • Pathways:large efforts across broad ranges of areas or approaches or users

  26. referenced items & collections referenced items & collections Special Databases Portals & Clients Portals & Clients Portals & Clients NSDL Services NSDL Services Other NSDL Services NSDL Collections NSDL Collections NSDL Collections Core Services: information retrieval CI Services browsing CI Services authentication Core Services: metadata gathering CI Services personalization Core Collection- Building Services protocols CI Services discussion Core Collection- Building Services harvesting CI Services annotation NSDL Information ArchitectureEssentially as developed by the Technical Infrastructure Workgroup User Interfaces CoreNSDL “Bus” Usage Enhancement Collection Building

  27. OCKHAM Library Network (NSDL)

  28. OCKHAM (Ming Luo) • Simplicity (a la OCCAM’s razor) • Support by Mellon and DLF • Four main ideas: • Components • Lightweight protocols • Open reference models (e.g., 5S, OAIS) • Community perspective and involvement • Funded by NSF in NSDL, with P2P, with Emory, Notre Dame, Oregon State, …

  29. OCKHAM Proposed Services • Alerting • Browsing • Cataloging • Conversion • OAI – Z39.50 • Pathfinding • Registry • (plus others such as from adapted ODL)

  30. Domain: graduate education, research Genre:ETDs=electronic theses & dissertations Submission: http://etd.vt.edu Collection: http://www.theses.org Project: Networked Digital Library of Theses & Dissertations (NDLTD) http://www.ndltd.org (supported by Ming Luo) A Digital Library Case Study

  31. OCLC SRU Interface => Dr. A.K. Tyagi

  32. ETD Union Search Mirror Site in China (CALIS)(http://ndltd.calis.edu.cn – popular site!)

  33. LOCKSS Extensions:Bing Liu, Xiaoyu Zhang, Ji-Sun Kim • Lots of copies keep stuff safe • Stanford (Vicky Reich) • Initial focus on lower levels, journals • Shift to OAI, esp. for ETDs • Collab with Emory (Martin Halbert) • NDIIP: AmericanSouth, MetaArchive • Help deploy and adapt, apply in other contexts • Another registry • Set of publisher manifests (information providers) • Set of storage systems (archival storage)

  34. Program Video Video Image Image Program Program Video Image XPMH 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 OA OA XPMH PMH OA XPMH OA XPMH XPMH OA XPMH OA Document Document Document XPMH XPMH 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 XPMH OA OA XPMH OA PMH XPMH Hussein Suleman(Capetown, S. Africa) open digital library

  35. Extended OAI-PMH Open Digital Library Protocol Protocol for Metadata Harvesting

  36. Extended OPEN ARCHIVE Open Digital Library Component OPEN ARCHIVE

  37. Open Digital Library Components • Running now • XML-File (data provider from file system) • Search: simple or in-memory (Essex) or generalized • Union, browse, recent, filter • E-journal/review, Submit, Edit, Annotation • Recommender, Rating; Mirroring (see JCDL’02) • Working with NCSA: from DB, unstructured text • Others in process • Classification/categorization • Registry (and other connections with web services)

  38. ETD-2 ETD-4 Video ETD-3 Image Program Program Video Image 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 ETD-1 Document Document 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 1010100101010010101010010101010101010101 Example Open Digital Library ODLRecent USER INTERFACE Recent PMH ODLUnion Filter PMH ODLUnion Union Browse PMH ODLBrowse PMH ODLUnion Filter PMH Search ODLSearch ETD DL for the Networked Digital Library of Theses and Dissertations (www.ndltd.org) Students and researchers ETD collections

More Related