1 / 20

A GANGA tutorial

A GANGA tutorial. Professor Roger W.L. Jones Lancaster University. Introduction. This tutorial gives a short introduction to using the Grid with GANGA GANGA = user front end GANGA is used by several communities, and by ATLAS and LHCb - may be useful for your other local users

silas-warner
Download Presentation

A GANGA tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A GANGA tutorial Professor Roger W.L. Jones Lancaster University

  2. Introduction • This tutorial gives a short introduction to using the Grid with GANGA • GANGA = user front end • GANGA is used by several communities, and by ATLAS and LHCb - may be useful for your other local users • You will submit some simple tasks to the Grid with GANGA • You will see a demonstration of running ATLAS software on the Grid using GANGA • Those with ATLAS softare set up can try this • This tutorial is based on the material here: https://twiki.cern.ch/twiki/bin/view/Atlas/GangaTutorial44 I invite you to try all the tutorial material offline. The tutorial normally runs over 2 days, so there is a lot in it, but it is well documented

  3. Ganga 3

  4. Grid Infrastructure WLCG based on EGEE and OSG Grid infrastructure. NDGF EGEE OSG GridPP WLCG 4

  5. Applications and Back ends LHCb Experiment neutral ATLAS AthenaMC (Production) Athena (Simulation/Digitisation/ Reconstruction/Analysis) Gauss/Boole/Brunel/DaVinci (Simulation/Digitisation/ Reconstruction/Analysis) Executable Local PBS LSF OSG PANDA LHCb WMS US-ATLAS WMS Implemented Work in progress In test/works • Running of a particular Application on a given Backend is enabled by implementing an appropriate adapter component or Runtime Handler • Can often use same Runtime Handler for several Backend: less coding

  6. Ganga - User Interface Scriptor Job details Logical Folders Job Monitoring Job builder Log window GangaGUI There is also a scripting interface (like pAthena) I will use the line mode……

  7. Step 1 - Grid Certificate into LXPLUS (I) • To submit Grid jobs you need a Grid certificate • We will be using the CERN LXPLUS User Interface so your certificate must be installed on LXPLUS • Go to the browser of the machine into which you retrieved your Grid Certificate • Find the “Certificates” dialogue box - often in “Preferences”, then “Advanced” • Click on “Export Certificate” or “Export Key Pair” • Choose a location on the local machine and click “OK” • Open a terminal window • Go to the location of the file you just exported, which will end with .pfx or .p12 • Copy the file to your LXPLUS account: • scp myfile.p12 username@lxplus.cern.ch:/afs/cern.ch/user/<letter>/<username>/. • Log into your lxplus account……

  8. Step 1 - Grid Certificate into LXPLUS (II) • mkdir .globus • mv myfile.p12 .globus/. • cd .globus • openssl pkcs12 -in myfile.p12 -clcerts -nokeys -out usercert.pem • openssl pkcs12 -in myfile.p12 -nocerts -out userkey.pem • chmod 400 userkey.pem • chmod 444 usercert.pem (unless you have one already)

  9. Step 2 - Make an executable file and test it • cd $HOME • Start up your favourite text editor • Copy in these lines and save the file (call it myscript.sh) • Test it: source myscript.sh #!/bin/sh echo 'myscript.sh running...’ echo "----------------------” /bin/hostname echo "----------------------” env

  10. Step 3: Start up GANGA • source /afs/cern.ch/sw/ganga/etc/setup-atlas.sh • ganga • Then enter your Grid Certificate password

  11. Step 4: Set up and run a LOCAL job • j=Job() • j.application=Executable() • j.application.exe=File('$HOME/myscript.sh') • j.backend=Local() • j.submit()

  12. Step 5: monitoring • To see all your jobs: jobs • To the status of a single job: jobs(number).status() • To see the details of a single job: jobs(number)

  13. Step 6: collect your results • Exit ganga: CTRL+D • cd $HOME/gangadir/Local/number/output • Open the file stdout with your favourite editor. This should show the output from your script. • You can look at the files from within GANGA as well: • !less $j.outputdir/stdout (note that ‘!’ means ‘execute this command in the shell, not GANGA)

  14. Step 7: Copy the job and switch to LCG mode • Start up GANGA again • jobs(number).copy() • jobs(number+1).backend=LCG() • jobs(number+1).submit() • Do your monitoring, etc again • When the job status is “completed”, check the output file again • In general this is how you should work; set up and test a LOCAL job, then copy and switch to LCG only when it is working • You can see where in the world the job ran by typing jobs(number) and looking in the back-end section of the report (under ‘CE’)

  15. Step 8: Try the GANGA graphical interface • The GANGA graphical interface (GUI) can do everything that the command line can do, but allows you to point and click • You can only use it if x-forwarding is set up on your terminal; to test this type xclock; you should see a clock on the screen. • To open the GUI, quit GANGA and re-start but with an additonal --gui option after ganga ganga --gui • Can you see your two jobs? • Can you submit a new job through the GUI?

  16. Help on GANGA • Several Ganga-related mailing lists have been set up, as detailed below. Most of these are managed within the framework of the CERN Simba ListBox Service, so that you can subscribe using the Simba web interface, and to access the archives you may need to give your username and password for the CERN Mail Server. The ATLAS HyperNews list is an exception, and to subscribe to this, or access the forum, you need to have registered using the ATLAS HyperNews New Membership page. General information on Ganga project • project-ganga@cern.ch List for Ganga developers • project-ganga-developers@cern.ch ATLAS HyperNews forum for Ganga user support • hn-atlas-GANGAUserDeveloper@cern.ch

  17. What’s next • A demonstration of how to run the ATLAS software framework in GANGA

  18. An Athena Demonstration • This assumes you are in the ATLAS VO, your cmt area set up and have checked out the examples into a test area….. source cmthome/setup.sh -tag=13.0.30,32 cd $TestArea/PhysicsAnalysis/AnalysisCommon/UserAnalysis/cmt cmt config source /afs/cern.ch/sw/ganga/install/etc/setup-atlas.csh Ganga #This is a temporary fix because the CERN srm was renamed without warning!!!!!* config['Athena']['LCGOutputLocation']='srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/scratch/Roger.Jones/ganga' #done - at least you see how to change config options - you can #also edit your .gangarc file to do the same thing….. j = Job() j.name='your_name'

  19. An Athena Demonstration j.application=Athena() j.application.prepare(athena_compile=False) j.application.option_file='/afs/cern.ch/user/r/rjones/testarea/AtlasOffline-13.0.30/PhysicsAnalysis/AnalysisCommon/UserAnalysis/run/AnalysisSkeleton_topOptions.py' j.application.max_events='100' j.inputdata=DQ2Dataset() j.inputdata.dataset="fdr08_run1.0003070.StreamMuon.merge.AOD.o1_r12_t1" j.outputdata=ATLASOutputDataset() j.outputdata.outputdata=['AnalysisSkeleton.aan.root'] j.backend=LCG() #Just in case you need to force a job to a specific site….. #j.backend.requirements.sites= ['LANCS'] j.submit()

  20. An Athena Demonstration To monitor jobs: jobs To execute as a python script: execfile('/afs/cern.ch/user/r/rjones/testarea/myscript.py') When finished: jobs(jobid).outputdata.retrieve()

More Related