1 / 31

Software and Data Challenge

Software and Data Challenge. Introduction Software Progress Optics/Analysis CKOV SciFi Reconstruction Event Display Materials/Physics Testing Documentation Professor’s Guide A Recent New User’s Result Data Challenge GRID / MICE VO Plans. Introduction.

summer
Download Presentation

Software and Data Challenge

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software and Data Challenge • Introduction • Software Progress • Optics/Analysis • CKOV • SciFi Reconstruction • Event Display • Materials/Physics • Testing • Documentation • Professor’s Guide • A Recent New User’s Result • Data Challenge • GRID / MICE VO • Plans M. Ellis - MICE CM17 - 24th February 2007

  2. Introduction • First half of talk will try to list highlights of the progress made since the last CM. • Second half will concentrate on the use of the GRID and the Data Challenge. • As with other recent talks I’ve given I will try to kill two birds with one stone and use writing the talk as a way to force myself to write a bit more documentation that will eventually find its way to the software section of the MICE website: • http://www.mice.iit.edu/software/software.html M. Ellis - MICE CM17 - 24th February 2007

  3. Software Progress • Continuing to make a lot of progress. • Focus is now mostly on fine tuning and polishing the existing tools, rather than writing new ones or adding major new features. • Very pleased to report that in the past few months several new users of G4MICE have started to use the software and hopefully this trend will continue. • Two areas of major attention have been: • Continued development of testing • Sorting out GRID issues so that the Data Challenge can be launched M. Ellis - MICE CM17 - 24th February 2007

  4. Optics/Analysis • Chris has already presented results at this CM and past Analysis meetings produced with the tools he has developed in G4MICE. • One new tool is an application that allows the user to specify a desired matched beam into the experiment. The application will generate a file containing a set of random particles that have the requested properties. • In the past (e.g. tracker validation) this was only possible by manual calculation of a covariance matrix and feeding the matrix into the Analysis application, it has now been made much more user-friendly. MiceModel MICEStage6/FullLattice.dat numEvts 100000 !the number of events to generate ZOffsetStart -5500. mm !the point at which the beam is generated InputBeamFileName for003.matched.dat !the name of the file that is generated ! NormalisedCanonicalAngularMomentum 0. !the Canonical angular momentum in the absorber LatticeStart -2750. mm !the start of the cooling channel lattice LatticeEnd 2750. mm !the end of the cooling channel lattice EnergyAtLatticeStart 226. MeV !energy of the reference particle at the lattice start TransverseEmittance 2.5 mm !transverse emittance of the beam at the start of the lattice LongitudinalEmittance 0.0001 ns !longitudinal emittance of the beam at the start of the lattice ! M. Ellis - MICE CM17 - 24th February 2007

  5. CKOV • Hideyuki has prepared a GEANT4 model of the CKOV, including the mirror shape. • In the process he identified a couple of bugs in GEANT4’s visualisation which were reported and one has already been fixed. • Focus is now shifting to implementing an accurate simulation of the Cherenkov light, which is non trivial (especially compared with sensitive detector code for scintillators, etc). • It is hoped that at some stage Yordan will add the CKOV to his newly acquired TOF responsibilities. M. Ellis - MICE CM17 - 24th February 2007

  6. SciFi Reconstruction • Aron has been working hard on writing a thorough set of tests to validate every stage of the SciFi reconstruction. • Tests of the Space point reconstruction, pattern recognition in a magnetic field and the Kalman fit using the Helix model are all written and are currently being used to find and fix the remaining bugs. • Aron’s focus in next week’s Software Workshop will be to fix the remaining issues in the space points and validate the track fit so that Tamas (new post doc at Imperial working part time on MICE) can use G4MICE to do the now urgent station spacing study (confirmation of spacing determined by Takashi last year). M. Ellis - MICE CM17 - 24th February 2007

  7. Event Display • Amongst his many other tasks, Michael has been writing code to visualise more classes using the G4MICE HepRep visualisation. • At the moment we have discovered a problem with the persistency of the MCParticle class which is under investigation. • Once this is fixed, he will carry on with the visualisation of the remaining reconstructed classes. M. Ellis - MICE CM17 - 24th February 2007

  8. Materials / Physics • Michael, Terry and Tamas have been looking at the physical properties of the materials that we get from the NIST database. • Several of them (in particular LH2!) have different properties to those in the PDG. • Tamas has started implementing additional materials in G4MICE that will have the PDG properties so that we can choose whichever we prefer and if desired look at the effect that these differences have. • Michael has implemented all of the MuScat targets and we will be working on some tests in the coming month to generate scattering distributions from GEANT4 so that they can be tested against the MuScat results. M. Ellis - MICE CM17 - 24th February 2007

  9. Testing • The number of tests continues to grow (currently over 100). • In addition, existing tests are being extended to test more variables (currently over 500), or configurations as appropriate. • Aron and Chris recently extended the Simulation to allow independent control over MCS and dEdx. This will be used as part of the new tests of the Kalman fit. • Many of the tests are producing PAW histograms (will eventually migrate to producing ROOT histograms) so not only can anyone in MICE run the tests to check that their copy of the software is performing correctly, but they can also produce the same plots shown in VCs or CMs (e.g. tracker performance) and check for themselves that the performance is as claimed. M. Ellis - MICE CM17 - 24th February 2007

  10. Documentation • Continues to lag development. • Doxygen documentation is generated from the source daily and can be found from the software website. • Gradually adding guides for new users as to how to use Data Cards, MiceModules, Materials, etc. • Professor’s Guide has been used successfully by many people (and very soon, a real live professor!) M. Ellis - MICE CM17 - 24th February 2007

  11. Professor’s Guide • Available from the software section of the MICE website: • http://www.mice.iit.edu/software/DataChallenge/Documentation/ProfessorsGuide.html • Contains step by step instructions for installing G4MICE from scratch on Linux, Mac OS X or Cygwin under Windows. • Successfully used by about 10 people (not all MICE) on all three platforms. • One unsuccessful attempt so far (Fedora Core 4), otherwise seems to work without much hassle. • I have a quick installation for people who are sharing a common file system on a cluster that will help get more people using it, let me know if you are interested! M. Ellis - MICE CM17 - 24th February 2007

  12. A Recent New User’s Results • With Aron’s help Terry has run a Simulation of all of the MICE steps that involve a tracker or two. • He put in a set of particles from TURTLE provided by Kevin Tilley and simulated them passing through each step. • They then wrote a small application that read in the simulated hits and produced histograms of the distribution of arrival time in each of the 10 stations. • This information will be used to determine the range in delays that will be needed for the L1A signals that will go into the AFE boards for the tracker readout. • This was achieved over a day and a half while Terry was at Imperial after the tracker workshop! • Please don’t be scared to give it a go – there are a lot of analyses that are needed that can be achieved with a comparable level of effort. M. Ellis - MICE CM17 - 24th February 2007

  13. Data Challenge • Aim to use the MICE VO on the GRID to generate large statistics of Simulated data and analyse it • Two aspects: • Software group • Want to use it to test the tools on high statistics. • Ensure that we know how to use the GRID and can move large amounts of data around. • Use it as a way of training mice to use the software and analyse results. • Try to identify what is missing that we need before data taking begins. • Analysis and Tracker groups • Want to produce specific simulations/analyses that will give answers to important questions. M. Ellis - MICE CM17 - 24th February 2007

  14. GRID / MICE VO • MICE VO has existed for a while now but has not been used for a G4MICE activity until very recently. • Originally trying to use the GRID portal being developed at Imperial College. • Have had a number of problems, some of which are fixed others are still being debugged. • Have learned a lot and in particular have had a huge amount of help from Henry. • Yesterday I was able to submit two jobs onto the GRID under the MICE VO and each ran successfully! • The first ran a short (5 events) simulation of MICE step 6 and returned the output file. • The second generated a slightly longer (5000 events) simulation of the same configuration and afterwards ran an application to generate ROOT histograms and returned the ROOT file. M. Ellis - MICE CM17 - 24th February 2007

  15. Jargon Alert!!! • Certificate: A way of identifying who you are. You get a certificate from your institute and can then join one or more VOs. • VO: Virtual Organisation – something that has resources on the GRID. You need to belong to the MICE VO in order to submit jobs that will use MICE VO resources. • Proxy: a way of letting the GRID know who you are without entering your password all the time. • Resource Broker: A machine that knows about your VO and which can find you resources (computers to run your jobs). • Computing Element (CE): A machine on the GRID that can run your jobs. • Storage Element (SE): Disk space on the GRID where data can go. • User Interface (UI): Machines from which you can submit GRID jobs (e.g. CSF at RAL). M. Ellis - MICE CM17 - 24th February 2007

  16. MICE VO • MICE currently has a resource broker at Imperial College: > lcg-infosites --vo mice rb gfe01.hep.ph.ic.ac.uk:7772 • There are Computing Elements in a number of UK institutes: > lcg-infosites --vo mice ce #CPU Free Total Jobs Running Waiting ComputingElement ---------------------------------------------------------- 52 1 0 0 0 dgc-grid-44.brunel.ac.uk:2119/jobmanager-lcgpbs-mice 3 3 0 0 0 dgc-grid-35.brunel.ac.uk:2119/jobmanager-lcgpbs-short 3 3 0 0 0 dgc-grid-35.brunel.ac.uk:2119/jobmanager-lcgpbs-rampdown 3 3 1 1 0 dgc-grid-35.brunel.ac.uk:2119/jobmanager-lcgpbs-production 3 3 0 0 0 dgc-grid-35.brunel.ac.uk:2119/jobmanager-lcgpbs-mice 122 1 0 0 0 dgc-grid-40.brunel.ac.uk:2119/jobmanager-lcgpbs-mice 1596 664 4 4 0 ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-x86_64 1596 664 837 836 1 ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_long 1596 664 0 0 0 ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_short 140 1 0 0 0 ce1.pp.rhul.ac.uk:2119/jobmanager-pbs-micegrid M. Ellis - MICE CM17 - 24th February 2007

  17. Joining in the Fun! • At the moment the people I know of who are in the MICE VO are: • Malcolm Ellis, Aron Fish, Paul Hodgson & Maiko Takahashi • In addition the following have GRID certificates and will shortly join the VO: • Rikard Sandstrom & Roumen Tsenov • Don’t let them have all the fun. • First you need to get a GRID certificate. This is handled by your University or lab. If you don’t know who to contact let me know and I will try to help you find out. • Once you have a certificate you can request to join the MICE VO by going to this website: • https://voms.gridpp.ac.uk:8443/voms/mice/webui/request/user/create M. Ellis - MICE CM17 - 24th February 2007

  18. MICE Resources • The only MICE institute that is currently providing resources to our VO is Brunel! • Hopefully somebody from some of the other institutes will contact me to help remedy this situation... M. Ellis - MICE CM17 - 24th February 2007

  19. First G4MICE job on the MICE VO • Yesterday I was able to run a couple of G4MICE jobs on the GRID with a lot of help from Henry. • The following slides will take you through the process of setting up to run a job, submitting it, monitoring it and retrieving the results. • I was using the CSF cluster at RAL to submit the jobs. • Unfortunately this cluster does not know about the MICE VO at the moment (there is a ticket with them to change this but it has not been acted on in the last week). • There are also issues with the VOMS system at the moment which seems to be stopping me launch GRID jobs. This is also under investigation but thankfully hasn’t stopped this attempt as I’m only in one VO. M. Ellis - MICE CM17 - 24th February 2007

  20. Preparing the Job • To submit a job, you must generate a proxy and then submit a file which is in a Job Description Language. • This allows you to specify what the job should do, what files are required (in and out), what VO you want it to run in, how much CPU it requires, etc. • There are two “Sandboxes” for passing files to the job and retrieving files afterwards (e.g. standard output and error). • These sandboxes are not for passing large files (e.g. Simulation results for a large run). • For this exercise, I made the job retrieve a statically linked executable (or two) from the MICE web server and in future tests we will attempt to pass large Simulation output files to a Storage Element (SE) for later Digitisation, etc. M. Ellis - MICE CM17 - 24th February 2007

  21. [ VirtualOrganisation = "mice"; Executable = "runsim.csh"; Arguments = ""; StdOutput = "sim.out"; StdError = "sim.err"; InputSandbox = { "runsim.csh", "files.tgz", "cards.in" }; OutputSandbox = { "sim.out", "sim.err", "Sim.out.stage6.gz" }; RetryCount = 0; JobType = "normal"; Type = "Job"; ] [ VirtualOrganisation = "mice"; Executable = "runSimRoot.csh"; Arguments = ""; StdOutput = "sim.out"; StdError = "sim.err"; InputSandbox = { "runSimRoot.csh", "files.tgz", "cards.simroot.in" }; OutputSandbox = { "sim.out", "sim.err", "sample.root" }; RetryCount = 0; JobType = "normal"; Type = "Job"; Requirements = other.GlueCEPolicyMaxCPUTime > 60; ] sim.jdl sim_root.jdl M. Ellis - MICE CM17 - 24th February 2007

  22. #!/bin/csh tar xvzmf files.tgz wget http://www.mice.iit.edu/software/grid/Simulation.gz gunzip Simulation.gz chmod 755 Simulation wget http://www.mice.iit.edu/software/grid/g4emlow.tgz tar xvzmf g4emlow.tgz ls -ltrh setenv MICE_PATH ${PWD} setenv MICESRC ${PWD} setenv MICEFILES ${PWD}/FILES/ setenv MICEPLATFORM Linux-g++ setenv MICEROOT ${PWD} setenv G4LEDATA ${PWD}/G4EMLOW4.0 env ./Simulation cards.in ls -ltrh #!/bin/csh tar xvzmf files.tgz wget http://www.mice.iit.edu/software/grid/Simulation.gz gunzip Simulation.gz chmod 755 Simulation wget http://www.mice.iit.edu/software/grid/RootHistograms.gz gunzip RootHistograms.gz chmod 755 RootHistograms wget http://www.mice.iit.edu/software/grid/g4emlow.tgz tar xvzmf g4emlow.tgz ls -ltrh setenv MICE_PATH ${PWD} setenv MICESRC ${PWD} setenv MICEFILES ${PWD}/FILES/ setenv MICEPLATFORM Linux-g++ setenv MICEROOT ${PWD} setenv G4LEDATA ${PWD}/G4EMLOW4.0 env ./Simulation cards.simroot.in ls -ltrh ./RootHistograms Sim.out.stage6.gz ls -ltrh runsim.csh runSimRoot.csh M. Ellis - MICE CM17 - 24th February 2007

  23. !****General MiceModel MICEStage6/Stage6.dat ! numEvts 5 ! ZOffsetStart -15 m SigmaX 1.0 mm SigmaXPrime 0.0 SigmaYPrime 0.0 AverageMomentum 240 MeV DeltaEoverE 0.01 ! OutputFile Sim.out.stage6.gz !****General MiceModel MICEStage6/Stage6.dat ! numEvts 5000 ! ZOffsetStart -15 m SigmaX 1.0 mm SigmaXPrime 0.0 SigmaYPrime 0.0 AverageMomentum 240 MeV DeltaEoverE 0.01 ! OutputFile Sim.out.stage6.gz cards.in cards.simroot.in M. Ellis - MICE CM17 - 24th February 2007

  24. Generating a Proxy • This assumes that you’ve already exported your certificate and stored the usercert.pem and userkey.pem in the ~/.globus directory. > grid-proxy-init Your identity: /DC=org/DC=doegrids/OU=People/CN=Malcolm Ellis 380333 Creating proxy .................................. Done Your proxy is valid until: Sun Feb 25 10:57:26 2007 • Proxies should be destroyed when you are done: > grid-proxy-destroy M. Ellis - MICE CM17 - 24th February 2007

  25. Match Resources to the Job • It is possible to find out what resources on your VO will be willing to accept your job (based on the requirements that you’ve listed): > edg-job-list-match --config ../gridpp_wl_cmd_var_mice.conf --config-vo ../gridpp_wl_vo_mice.conf sim.jdl Selected Virtual Organisation name (from --config-vo option): mice Connecting to host gfe01.hep.ph.ic.ac.uk, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_short ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-x86_64 ce1.pp.rhul.ac.uk:2119/jobmanager-pbs-micegrid dgc-grid-35.brunel.ac.uk:2119/jobmanager-lcgpbs-mice dgc-grid-35.brunel.ac.uk:2119/jobmanager-lcgpbs-short dgc-grid-40.brunel.ac.uk:2119/jobmanager-lcgpbs-mice dgc-grid-44.brunel.ac.uk:2119/jobmanager-lcgpbs-mice ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_long *************************************************************************** M. Ellis - MICE CM17 - 24th February 2007

  26. Submit the Job • The job submission, if successful, will return a unique reference to the job which you can then use to track its progress and ultimately retrieve the output of the job. > edg-job-submit --config ../gridpp_wl_cmd_var_mice.conf --config-vo ../gridpp_wl_vo_mice.conf sim.jdl Selected Virtual Organisation name (from --config-vo option): mice Connecting to host gfe01.hep.ph.ic.ac.uk, port 7772 Logging to host gfe01.hep.ph.ic.ac.uk, port 9002 ********************************************************************************************* JOB SUBMIT OUTCOME The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw ********************************************************************************************* M. Ellis - MICE CM17 - 24th February 2007

  27. Tracking the Job • The command edg-job-status will allow you to watch as the job is Scheduled, starts Running and eventually is Done: > edg-job-status https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw Current Status: Scheduled Status Reason: Job successfully submitted to Globus Destination: ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_short reached on: Sat Feb 24 14:34:55 2007 ************************************************************* > edg-job-status https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw Current Status: Running Status Reason: Job successfully submitted to Globus Destination: ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_short reached on: Sat Feb 24 14:37:20 2007 ************************************************************* > edg-job-status https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: ce02.esc.qmul.ac.uk:2119/jobmanager-lcgpbs-lcg2_short reached on: Sat Feb 24 14:43:35 2007 ************************************************************* M. Ellis - MICE CM17 - 24th February 2007

  28. Get the Results • The files that were specified in the Output SandBox can be retrieved with the command edg-job-get-output: > edg-job-get-output https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw Retrieving files from host: gfe01.hep.ph.ic.ac.uk ( for https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - https://gfe01.hep.ph.ic.ac.uk:9000/UC1jyoUx_65IuA3goKqxsw have been successfully retrieved and stored in the directory: /tmp/jobOutput/ellis_UC1jyoUx_65IuA3goKqxsw ********************************************************************************* M. Ellis - MICE CM17 - 24th February 2007

  29. Simulation + ROOT Analysis • In the second job, a beam is simulated going through MICE Step VI, with 5000 events being generated. • Once the Simulation was done, another G4MICE application was run which read in the Simulation results and generated some histograms from the hits in the SciFi tracker and stored them in a ROOT file. • The ROOT file was then returned. • This serves as a model for the GRID version of the kind of analysis that Aron and Terry have done interactively and which requires minimal programming effort on the part of new users. M. Ellis - MICE CM17 - 24th February 2007

  30. What is Missing? • I need to get a little more familiar with the GRID tools, but aim to start launching jobs in earnest at next week’s Software Workshop. • Currently waiting for specific requests for beams/configurations to be studied from the Analysis group. • Also short of volunteers to analyse the simulations once they are done: • Tamas will do the station spacing work with Aron • Pietro has volunteered to look at the Analysis group related simulation. • Anyone else interested??? • John and his team are going to need some help, and it is a good way to start learning how to use these tools to perform an analysis! M. Ellis - MICE CM17 - 24th February 2007

  31. Plans • Henry has offered to help me understand the storage elements and how to transfer files around. • Once I have run a few more less trivial jobs and collected further software work at the workshop (improvements, bug fixes, etc). I will release a new tag. • This tag will be used to launch the first wave of the challenge. • In parallel Aron will be preparing the tools needed by Tamas to do the Station Spacing study. • I also want to launch a set of RF background simulations as soon as possible as we’ve not looked at this for a few years now and we will soon be getting some results from the MTA to compare them with. M. Ellis - MICE CM17 - 24th February 2007

More Related