1 / 8

Request Execution: a Tier 3 Perspective

Request Execution: a Tier 3 Perspective. Request. Use local resources (processing and storage) to apply program P to a subset of files from collection C that meets constraint Q Files must be staged locally No output Fixed amount of local disk space Has to compete for CPU power.

mimis
Download Presentation

Request Execution: a Tier 3 Perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Request Execution: a Tier 3 Perspective

  2. Request Use local resources (processing and storage) to apply program P to a subset of files from collection C that meets constraint Q • Files must be staged locally • No output • Fixed amount of local disk space • Has to compete for CPU power

  3. Expectations • No need to know location of C • Reliable • Error Reporting • Efficient • Progress Reporting

  4. Method Develop a Request Executor (ReqEx) and test its end-to-end functionality in a testbed with the following components: • Query Estimator (QE) at LBL • Meta Catalog (MCAT) provided by SRB • Collection replicated at PPDG sites (LBL,SDSC, ...) • File staging via SRB • Condor pool and disk cache in Wisconsin

  5. Execution Steps • Receive C and Q from user • Locate QE for C • Send Q to QE and get list of files • Obtain file replication information from MCAT • For each file • Select storage location • Allocate local space and stage file • Apply F to file • Remove file and free local space

  6. Building the Testbed • Install SRB and QE related software in Wisconsin • Obtain SRB account • Install SRB software in LBL • Register LBL PPDG collection in MCAT • Replicate (and register) PPDG collection at SDSC. • Establish disk cache at Wisconsin

  7. ReqEx prototype • Second version built with software components from Condor • Prototype works • Only uses the Sget command of SRB • Does not support replication • QE location hard coded.

  8. Main Lessons • Installation of software components can not be done by “the locals” • Lack of support for inter-component exception handling a major obstacle • Need for a “development” testbed and a “production” testbed • Software upgrades a problem.

More Related