1 / 26

Globus Data Replication Services

Globus Data Replication Services. Ann Chervenak, Robert Schuler USC Information Sciences Institute. Motivation for Data Replication Services. Data-intensive applications need higher-level data management services that integrate lower-level Grid functionality

selma
Download Presentation

Globus Data Replication Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Globus Data Replication Services Ann Chervenak, Robert Schuler USC Information Sciences Institute

  2. Motivation for Data Replication Services • Data-intensive applications need higher-level data management services that integrate lower-level Grid functionality • Efficient data transfer (GridFTP, RFT) • Replica registration and discovery (RLS) • Eventually validation of replicas, consistency management, etc. • Goal is to generalize the custom data management systems developed by several application communities • Eventually plan to provide a suite of general, configurable, higher-level data management services • Globus Data Replication Service (DRS) is the first of these services

  3. The Data Replication Service • Included in the Tech Preview of GT4.0 release • Design is based on the publication component of the Lightweight Data Replicator system • Developed by Scott Koranda from U. Wisconsin at Milwaukee • Functionality • Replicate a set of files in the Grid on a local site • Users identify a set of desired files • DRS queries Replica Location Service to discover current locations of these files • Creates local replicas of desired files using the Reliable File Transfer Service • Registers new replicas in Replica Location Service for discovery

  4. Outline • Terminology • Functionality of Data Replication Service • Background: Components used by DRS • Replica Location Service • GridFTP Data Transport protocol • Reliable File Transfer Service • DRS Design • Implementation in GT4 environment • Evaluation of DRS performance in a wide area Grid • Future work

  5. Some Terminology • A logical file name (LFN) is a unique identifier for the contents of a file • Typically, a scientific collaboration defines and manages the logical namespace • Guarantees uniqueness of logical names within that organization • A physical file name (PFN) is the location of a copy of the file on a storage system. • The physical namespace is managed by the file system or storage system • For example, the LIGO environment currently contains: • More than six million unique logical files • More than 40 million physical files stored at ten sites

  6. DRS Overview • Client uses DRS interface to specify which files are required at local site • DRS uses: • Delegation Service to delegate proxy credentials • Globus RLS to discover whether replicas exist locally and where they exist in the Grid • Selection algorithm to choose among available source replicas • Globus Reliable File Transfer service to copy data to local site • This uses GridFTP data transport protocol • Globus RLS to register new replicas

  7. Background: The Replica Location Service • A Replica Location Service (RLS) is a distributed registry that records the locations of data copies and allows replica discovery • RLS maintains mappings between logical identifiers and target names • Must perform and scale well: support hundreds of millions of objects, hundreds of clients • E.g., Laser Interferometer Gravitational Wave Observ. • RLS servers at 8 sites • Maintain associations between 6 million logical file names & 40 million physical file locations

  8. RLS Features Replica Location Indexes RLI RLI • Local Replica Catalogs (LRCs) contain consistent information about logical-to-target mappings LRC LRC LRC LRC LRC Local Replica Catalogs • Replica Location Index (RLI) nodes aggregate information about one or more LRCs • LRCs use soft state update mechanisms to inform RLIs about their state: relaxed consistency of index • Optional compression of state updates reduces communication, CPU and storage overheads

  9. Background: GridFTP • A secure, robust, fast, efficient, standards based, widely accepted data transfer protocol • Features: • Standard FTP: get/put etc., 3rd-party transfer • GSS binding, extended directory listing, simple restart • Striped/parallel data channels • Partial file • TCP buffer setting • Progress monitoring, extended restart • The Globus Toolkit supplies a reference implementation: • Server • Client tools (globus-url-copy) • Development Libraries

  10. Control Control Data Data Background: Reliable File Transfer Service • RFT accepts SOAP description of transfer • Writes state to a database • Uses Java GridFTP client library to initiate 3rd part transfers • Restart Markers stored in the database • Allow for restart in the event of RFT failure • Supports concurrency, i.e., multiple files in transit • Check status: • Subscribe to notifications • Poll for status RFT Client SOAP Messages Notifications(Optional) RFT Service

  11. DRS Functionality • Initiate a DRS Request • Create a delegated credential • Create a Replicator resource • Monitor Replicator resource • Discover replicas of desired files in Replica Location Service, select among replicas • Transfer data to local site with Reliable File Transfer Service • Register new replicas in RLS catalogs • Allow client inspection of DRS results • Destroy Replicator resource DRS implemented in Globus Toolkit Version 4, complies with Web Services Resource Framework (WS-RF)

  12. Relationship to Other Globus Services At requesting site, deploy: • WS-RF Services • Data Replication Service • Delegation Service • Reliable File Transfer Service • Pre WS-RF Components • Replica Location Service (Local Replica Catalog, Replica Location Index) • GridFTP Server

  13. Service State Management: Resource Resource Property State Identification: Endpoint Reference State Interfaces: GetRP, QueryRPs, GetMultipleRPs, SetRP Lifetime Interfaces: SetTerminationTime ImmediateDestruction Notification Interfaces Subscribe Notify ServiceGroups Resource RPs WSRF in a Nutshell Service GetRP GetMultRPs EPR EPR EPR SetRP QueryRPs Subscribe SetTermTime Destroy

  14. Credential RP Service Container Create Delegated Credential Data Rep. Client • Create delegated credential resource • Set termination time Replica Catalog Replica Index • Credential EPR returned RFT EPR Replica Catalog proxy Replica Catalog Replica Catalog Delegation • Initialize user proxy cert. GridFTP Server GridFTP Server MDS

  15. Replicator Credential RP RP Service Container Create Replicator Resource Data Rep. Client EPR Replica Catalog Replica Index • Create Replicator resource • Pass delegated credential EPR • Set termination time RFT Replica Catalog Replica Catalog Replica Catalog • Replicator EPR returned Delegation • Access delegated credential resource GridFTP Server GridFTP Server MDS

  16. Index Replicator Credential RP RP RP Service Container Monitor Replicator Resource Data Rep. Client Replica Catalog Replica Index • Subscribe to ResourceProperty changes for “Status” RP and “Stage” RP RFT Replica Catalog Replica Catalog Replica Catalog Delegation • Add Replicator resource to MDS Information service Index EPR • Periodically polls Replicator RP via GetRP or GetMultRP GridFTP Server GridFTP Server MDS • Conditions may trigger alerts or other actions (Trigger service not pictured)

  17. Index Replicator Credential RP RP RP Service Container Query Replica Information Data Rep. Client Replica Catalog • Notification of “Stage” RP value changed to “discover” Replica Index RFT Replica Catalog • Replicator queries RLS Replica Index to find catalogs that contain desired replica information Replica Catalog Replica Catalog • Replicator queries RLS Replica Catalog(s) to retrieve mappings from logical name to target name (URL) Delegation GridFTP Server GridFTP Server MDS

  18. Credential Transfer Index Replicator RP RP RP RP Service Container Transfer Data • Periodically poll “ResultStatus” RP via GetRP • When “Done”, get state information for each file transfer Data Rep. Client Replica Catalog EPR • Notification of “Stage” RP value changed to “transfer” Replica Index EPR RFT Replica Catalog • Data transfer between GridFTP Server sites • Create Transfer resource • Pass credential EPR • Set Termination Time • Transfer resource EPR returned Replica Catalog Replica Catalog Delegation GridFTP Server GridFTP Server • Access delegated credential resource MDS • Setup GridFTP Server transfer of file(s)

  19. Credential Transfer Index Replicator RP RP RP RP Service Container Register Replica Information Data Rep. Client Replica Catalog • Notification of “Stage” RP value changed to “register” Replica Index RFT Replica Catalog Replica Catalog Replica Catalog • Replicator registers new file mappings in RLS Replica Catalog Delegation • RLS Replica Catalog sends update of new replica mappings to the Replica Index GridFTP Server GridFTP Server MDS

  20. Transfer Index Replicator Credential RP RP RP RP Service Container Client Inspection of State Data Rep. Client Replica Catalog • Client inspects Replicator state information for each replication in the request • Notification of “Status” RP value changed to “Finished” Replica Index RFT Replica Catalog Replica Catalog Replica Catalog Delegation GridFTP Server GridFTP Server MDS

  21. Transfer Index Replicator Credential RP RP RP RP Service Container Resource Termination TIME Data Rep. Client Replica Catalog • Termination time (set by client) expires eventually Replica Index RFT Replica Catalog Replica Catalog Replica Catalog • Resources destroyed (Credential, Transfer, Replicator) Delegation GridFTP Server GridFTP Server MDS

  22. Performance Measurements: Wide Area Testing • The destination for the pull-based transfers is located in Los Angeles • Dual-processor, 1.1 GHz Pentium III workstation with 1.5 GBytes of memory and a 1 Gbit Ethernet • Runs a GT4 container and deploys services including RFT and DRS as well as GridFTP and RLS • The remote site where desired data files are stored is located at Argonne National Laboratory in Illinois • Dual-processor, 3 GHz Intel Xeon workstation with 2 gigabytes of memory with 1.1 terabytes of disk • Runs a GT4 container as well as GridFTP and RLS services

  23. DRS Operations Measured • Create the DRS Replicator resource • Discover source files for replication using local RLS Replica Location Index and remote RLS Local Replica Catalogs • Initiate an Reliable File Transfer operation by creating an RFT resource • Perform RFT data transfer(s) • Register the new replicas in the RLS Local Replica Catalog

  24. Experiment 1: Replicate 10 Files of Size 1 Gigabyte Component of Operation Time (milliseconds) Create Replicator Resource 317.0 Discover Files in RLS 449.0 Create RFT Resource 808.6 Transfer Using RFT 1186796.0 Register Replicas in RLS 3720.8 • Data transfer time dominates • Wide area data transfer rate of 67.4 Mbits/sec

  25. Experiment 2: Replicate 1000 Files of Size 10 Megabytes Component of Operation Time (milliseconds) Create Replicator Resource 1561.0 Discover Files in RLS 9.8 Create RFT Resource 1286.6 Transfer Using RFT 963456.0 Register Replicas in RLS 11278.2 • Time to create Replicator and RFT resources is larger • Need to store state for 1000 outstanding transfers • Data transfer time still dominates • Wide area data transfer rate of 85 Mbits/sec

  26. Future Work • Continued performance testing of DRS: • Increasing the size of the files being transferred • Increasing the number of files per DRS request • Add and refine DRS functionality as needed by GEON and other applications • E.g., add a push-based replication capability • Add fine-grained authorization capability to RLS, DRS • Long-term: • Will develop a suite of general, configurable, composable, high-level data management services

More Related