Grid Services for Digital Archive

1. Grid Services for Digital Archive Tao-Sheng Chen Academia Sinica Computing Centre doug@gate.sinica.edu.tw

2. Content • Introduction to Grid • Demands from Digital Archive • DAGS Infrastructure and Architecture • MSC ( Metadata & Storage Controller) • DORE Grid Service • Geospatial Grid Service • Summary

3. What is Grid • http://gridcafe.web.cern.ch/gridcafe/ • the Grid is a service for sharing computer power and data storage capacity over the Internet. The Grid goes well beyond simple communication between computers, and aims ultimately to turn the global network of computers into one vast computational resource (Web is a service for sharing information over the Internet)

4. Important Issues of NDAP • Intellectual property rights 􀂾 • Time, Space and Language Coordination􀂾 • Multi-lingual issues (encoding)􀂾 • Public information systems􀂾 • Meta-language and Documentation􀀹 • Metadata 􀀹 • Content Markup􀀹 • References and Linking􀂾 • Dissemination and Sharing􀂾 • Cooperation and collaboration􀂾 • Scalability, Adaptability and Durability

5. Demands of Digital Archive • Persistent digital objects, 􀂾 • Well-organized information structure for effective content management􀂾 • Efficient and accurate information retrieval mechanism􀂾 • Flexible services for variant users needs􀂾 • Consistency􀂾 • Integrate relationship between information management and data management􀂾 • High-performance remote data access􀂾 • Authentication and authorization􀂾 • Resource discovery and monitoring

6. What we shall supply • Metadata System Integration • Reliable and efficient storage system􀀹 • Reliable replication system -> replica locating mechanism􀀹 • Reduced query latency ->query routing scheme􀀹 • Load sharing􀀹 • Robust, high availability􀀹 • Manageability􀀹 • High Throughput􀀹 • Adaptive􀀹 • Transparency of location and protocol􀀹

7. Challenge • Big Challenge of IT for cataloging, searching, retrieval, management, identification, knowledge discovery, and integration • Integration and Retrieval of Information Resources

8. Approach • Develop Grid Services that can integrate heterogeneous metadata systems, distributed database management systems and geospatial information systems. • Provide a framework to exchange different metadata XML documents (EAD, DC, FGDC …) in “National Digital Archives Program”.

9. Digital Archive Grid Service Infrastructure User Requests Digital Archive Portal XML Data XML Data Participant Node Participant Node Aggregated Data Aggregated Data Service Metadata Service Metadata Detailed Object Data Service Metadata Detailed Object Data Object Index Data Service Metadata Data Grid Nodes

10. Building Grid Service for DORE • DORE (Document REtrieval) is • A middleware • A library • A tool • for programmers to develop metadata database applications • DORE is a tool in Open Digital Archive Environment (ODAE). • Migrate DORE applications to Grid Infrastructure , and also have backward compatibility to existing system.

12. The UMLof DORE Grid Service

13. Dore Grid Service GUI Client

14. Next Steps of DORE Grid • Security Issue • Add CA authority in framework and archieveinter-organizational data sharing. • Security management. • Deploy DORE Grid Service in organizations • Other organization could build their own client application to use this framework • DORE Grid systems deploy in orgs. • Data sharing between organizations. • WSRF & GT4 ?

15. Geospatial Grid Service • Three basic categories of GIS Grid Services: • Data Services • Processing Services • Catalog Services

16. Services Architecture Approach For Example… Applications e.g. Historical research planning, Administrative boundary Changes Create map Find a historical map? Find place names in Qing Dynasty? Users uses Metadata Service, Gazetteer Service, Web map service Services e.g., Metadata Service, Gazetteer service, Web Map Service Other Applications Data e.g., topographic, thematic, imagery, toponymy, metadata based on Base historical maps, Geographical Names, Map features

17. GGS GGS DGS DGS Catalog Catalog data metadata data metadata geodata geodata other data other data Distributed Spatial Data Infrastructure Environment web client Symbol Key Service Catalog DGS DOREGrid service interface Project Site • Search Gateway • multiviewer application server GGS GeospatialGrid services interface WMS DGS register clients other distributed servers Node 1 Node 2 Node 3 GGS data metadata geodata other data

18. Metadata Service for Geospatial Data

19. MSC • MetaData & Storage controller

20. Plan the strategy DAGS is evolving to be an interoperable network of databases and information technology tools using Web services and Grid technologies. • In the near term, DAP will provide a national metadata registry of the available data with open interfaces through Grid service . • Building on the contents of this registry, DAGS will provide its own central portal that enables simultaneous queries against different databases held by distributed, even worldwide sources. • In the long term, different level objects can be linked to the system. • These will facilitate and enable data mining of unprecedented utility and e-Science.

21. Summary • Achievements 1. The Grid services cooperate with Geospatial Information system was developed and tested. 2.The DORE Grid middleware was implemented and rebuilded. 3. The metadata register of different provider and databases were completed. 4. Prototype of MSC (Metadata & Storage Controller) • Future Works 1.Integrate heterogeneous storage system and metadata system. 2.Refining the technologies of Data Grid 3.Developing the knowledge and e-Science discovery

22. Question? (demo)