1 / 9

Data Grid Federation by RNS

Data Grid Federation by RNS. GFS-WG, OGF23 Balcelona. Hideo Matsuda Osaka University / NAREGI. Use of Metadata in e-Science. For scientists, not only data itself but also its metadata is very important. Example of metadata: data ID, locations of data,

river
Download Presentation

Data Grid Federation by RNS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Grid Federation by RNS GFS-WG, OGF23 Balcelona Hideo Matsuda Osaka University / NAREGI

  2. Use of Metadata in e-Science • For scientists, not only data itself but also its metadata is very important. • Example of metadata: data ID, locations of data, experiment materials, instruments, and methods • Relationships between data are described as metadata. 2

  3. Example of Metadata • Metadata is often described with hierarchical representation in many sciences. Nucleotide Protein ATLAS CMS Bacteria Sequence Structure 20071003 20080110 Plant Primate Vertebrate run1 run2 sp|P37231 pdb|1FM6 gb|AY157024 track1 track2 High Energy Physics Molecular Biology 3

  4. Metadata Management using File Catalog • Currently metadata are mainly stored in File Catalogs using their hierarchical namespace functionality. • gLite: LFC, Fireman • iRODS (SRB): MCAT • Globus: RLS • NAREGI: Gfarm • File Catalog information of different Grid middlewares do not have compatibility to each other. • It is not easy to exchange metadata over different Grid middlewares. 4

  5. Resource Namespace Service (1) http://www.ogf.org/documents/GFD.101.pdf 5 RNS lets you map any resource into single, hierarchical namespace Resources are referred to in a form of EndpointReference (WS-Addressing) RNS Specification is published as GFD-R-P.101 RNS implementation is available from U.Virginia and U.Tsukuba.

  6. Resource Namespace Service (2) Hierarchical namespace management that provides name-to-resource mapping Basic Namespace Component Virtual Directory Non-leaf node in hierarchical namespace tree Junction Name-to-resource mapping that interconnects a reference to any existing resource into hierarchical namespace /grid ogf jp file1 file2 data gfs file2 file1 file3 file4 EPR2 EPR1 EPR: Endpoint Reference 6

  7. Comparison RNS with File Catalog • GFS Naming Profile on top of RNS and File Catalog Service are basically the same • File Catalog implies loosely coupled federation, whereas File System Directory implies rather tightly coupled federation • File Catalog Standardization is required by many parties 7

  8. Data Grid Federation with RNS (Plan) • RNS can interconnect a reference to any existing resource into hierarchical namespace • Most of Grid middlewares have GridFTP for data transfer gUse RNS as a (standardized?) File Catalog Use GridFTP URL “gsiftp://.../” as the address of Endpoint Reference. (1) query RNS Client (2) EPR list (including address) (3) Access with GridFTP protocol RNS Globus GridFTP Server gLite File Server (SRM) iRODS File Server NAREGI File Server (Gfarm) 8

  9. Summary • RNS has a potential functionality to federate Data Grids over different middlewares. • The federation encourages scientists to proceed international collaboration. • RNS implementation is available. It need to be re-designed towards production level. 9

More Related