1 / 22

Happy April Fools

Happy April Fools. Unfortunately, not yet …. … but I hope so one day. Introducing LCG to LHCb. Ian Stokes-Rees. Summary. Quick LCG Summary Getting started Accessing the LCG Job Submission on LCG Data Management on LCG References. LCG: The Good News.

tanner
Download Presentation

Happy April Fools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Happy April Fools Unfortunately, not yet … … but I hope so one day.

  2. Introducing LCG to LHCb Ian Stokes-Rees

  3. Summary • Quick LCG Summary • Getting started • Accessing the LCG • Job Submission on LCG • Data Management on LCG • References

  4. LCG: The Good News • 10,000+ CPUs all at your disposal, and constantly growing • Spread over Europe, North America and the rest of the world • Largest general purpose grid computing cluster in the world

  5. LCG: Key Features • Single sign-on via x509 PKI digital certificates • Uniform file access across all LCG sites • Ability to define multiple logical names for a single global file • Replication and auto-replication facilities • Automated scheduling of jobs to suitable resources • Based on job requirements • Based on resource availability/cluster loads • Based on input and output data location • Numerous API libraries to integrate LCG services into software

  6. LCG: The Reality • Hard to get started • x509 Digital Certificate takes time to acquire • Commands not obvious • System not always running 100%

  7. LCG: LHCb-Help • ESUS: Experiment Specific User Support • Ian Stokes-Rees • Joel Closier • ESUS rep 3??? • savannah.cern.ch/ projects/lhcb-help • Documentation • Support System • Links

  8. Summary of LCG Registration http://savannah.cern.ch/projects/lhcb-help Register on savannah.cern.ch Register for LHCb-Help Find your CA (Certificate Authority) Apply for x509 cert Meet your local RA (Registration Auth.) Receive CA signed cert Create browser and command-line versions of x509 cert Save cert onto USB key (both browser and CLI versions), and make scp’able Load cert into browser(s) Register your certificate with the LHCb VO (Virtual Organisation) Find a machine with the LCG User Interface Software installed All this is documented on the LHCb-Help Savannah site

  9. Getting Started: Digital Certificates • You must apply for an x509 digital certificate • Find your “local” CA e.g.: • CERN: service-grid-ca.web.cern.ch • UK e-Science: ca.grid-support.ac.uk • CNRS: igc.services.cnrs.fr/Datagrid-fr • It may take several days to receive your digital certificate, so please apply before you need it! • Walk-through on LHCb-Help website • Submit support request if you have problems

  10. LCG Demonstration

  11. Part 1: Review of Registration Savannah LHCb-Help CA Site CERN UK e-Science CNRS LCG/VO Registration Certificate: Data: Version: 3 (0x2) Serial Number: 1741 (0x6cd) Signature Algorithm: md5WithRSAEncryption Issuer: C=UK, O=eScience, OU=Authority, CN=CA/Email=ca-operator@grid-support.ac.uk Validity Not Before: Mar 5 17:41:30 2004 GMT Not After : Mar 5 17:41:30 2005 GMT Subject: C=UK, O=eScience, OU=Oxford, L=OeSC, CN=Ian Stokes-rees Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (1024 bit)

  12. Grid Topology Close SE WN GK LCG Site Storage Element Storage Element Worker Node SE RB UI Resource Broker User Interface Gate Keeper

  13. Part 2: Job Description Language Executable = "/bin/hostname"; Arguments = "-f"; StdOutput = "std.out"; StdError = "std.err"; OutputSandbox = {"std.out","std.err"}; • Other key JDL attributes: • InputSandbox, InputData, StdInput, OutputData • Environment • Requirements, Rank • See LCG-2 User Guide for more details

  14. Part 2: Submitting LCG Jobs grid-proxy-init grid-proxy-info edg-job-list-match hostname.jdl edg-job-submit hostname.jdl edg-get-status <job-id> edg-job-get-logging-info <job-id> edg-job-get-output <job-id>

  15. Part 2: Useful Additions • Options: --vo lhcb : usually required -i <jid_file> : list of job IDs -o <file> : output of command -r <resource> : send job to specific resource • .BrokerInfo file in Worker Node “home” directory with details of LCG site • edg-brokerinfo to extract information from .BrokerInfo file • Usual use case is to prepare a script which gets uploaded in InputSandbox and then executed

  16. Part 3: Data Management • Types of LCG Files: • file://<path> : local file • lfn:<string> : logical file name (LFN) • guid:<hash> : global unique file ID (GUID) • sfn://<path> : storage URL (SURL) • <protocol>://<path> : transfer URL (TURL)

  17. Part 3: Data Commands • Copying file to Grid Storage: edg-rm cr <file> -l lfn:<string> • Copies local file to grid storage and registers specified Logical File Name (LFN) • Returns the guid:<hash> • Copying file from Grid Storage: edg-rm cp <lfn> file://<path> • Finds file based on LFN and copies to specified path

  18. Part 3: Data Commands • Replicating Files: edg-rm rep <lfn> -d <replica_site> • Puts replica of file at specified site • Deleting Files: edg-rm del <lfn|guid> -s <site> • Deletes specified file from the specified site • If SURL specified, -s option not required • --all-available will remove all replicas

  19. Part 3: Data Commands • Linking – Adding an LFN: edg-rm aa <guid> <lfn> • Adds LFN for specified GUID • Accessing files from inside a job: • Use edg-rm cp <lfn|guid> file://<path> to access • Specify InputData in JDL, containing LFN or GUID • See edg-gridftp-* commands for direct control of files on SE • See LCG User Guide for more information

  20. LCG: What To Expect • Job Execution: • Currently looking at 6 minute overhead to process job completely • Data Operations: • 10-30 seconds per operation, on top of any file transfer • Long, Meaningless Strings: • Expect to be inundated with Job ID URLs, GUIDs, and LFNs • lsh (lhcb shell) may help, if I ever have the time to write it • will store all these items in a central DB with XML-RPC access and searchable by LCG identity

  21. LCG Key Links • LCG LHCb-Help (ESUS) • http://savannah.cern.ch/projects/lhcb-help • Grid Operations Centre • http://goc.grid-support.ac.uk

More Related