1 / 18

Ideas on the LCG Application Architecture

Ideas on the LCG Application Architecture. Application Architecture Blueprint RTAG 12 th June 2002 P. Mato / CERN. Application Domains. Event Processing Applications Trigger, Simulation, Reconstruction, Selection programs Data Analysis Event/Detector display, data presentation programs

liona
Download Presentation

Ideas on the LCG Application Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ideas on the LCG Application Architecture Application Architecture Blueprint RTAG 12th June 2002 P. Mato / CERN

  2. Application Domains • Event Processing Applications • Trigger, Simulation, Reconstruction, Selection programs • Data Analysis • Event/Detector display, data presentation programs • Detector calibration • Calibration and Alignment programs • Job configuration, submission, monitoring and control • Grid awareness Mainly batch oriented. Interactive for development & debugging. Real-time. Mainly interactive Batch and interactive Mainly interactive LCG Application Architecture

  3. No Disjoint Domains • For example, the LHCb requirements for interactive analysis: • Better [than PAW] integration with experiment framework • consistent with the analysis batch environment, use the same toolkits and experiment algorithms/tools • access the experiment data objects and allow browsing • integrated with event display • allow interactive reconstruction and simulation LCG Application Architecture

  4. Architectural styles • General categorization of systems [1] user-centric focus on the direct visualization and manipulation of the objects that define a certain domain data-centric focus upon preserving the integrity of the persistent objects in a system computation-centric focus is on the transformation of objects that are interesting to the system [1] G. Booch, “Object Solutions”, Addison-Wesley 1996 LCG Application Architecture

  5. Different style in different domains • The applications in the different domains may have different emphasis in: • Interactivity • Database • Computation • Elements of all three are present in all applications data- centric user- centric User Interface Interactivity Scripting Object store Database Data Integrity Algorithms Computation computation-centric LCG Application Architecture

  6. Computation-centric: GAUDI • Framework adequate for “all” event processing applications • Algorithms process “event data” with the help of “services” and using “detector data”. LCG Application Architecture

  7. User-centric: PAW • Provides interactive graphical presentation and statistical or mathematical analysis, working on objects familiar to physicists like histograms, event files (Ntuples), vectors, etc. • Is based on several components of the CERN Program Library. Easy to use and freely available • No integration within experiments framework • Offline analysis done in 2 separate steps “batch” and “interactive”, the user decides how much to do in each step LCG Application Architecture

  8. The Challenge • Many components are shared among the different applications in the different domains • We would like to profit from the best of the existing solutions (implementations) for each component but not duplicate everything. >>> INTEGRATION <<< LCG Application Architecture

  9. Component interface specifications • Integration technologies • Mainly for distributed computing • CORBA, DCOM (.NET), Web Services, JavaBeans • Abstract interfaces • C++ “in process” integration • Robust and efficient. A bit awkward for end-users. • Scripting extension modules • Ideal for prototyping. Rapid application development (RAD). • Dependent of the scripting tool Probably the best solution but unrealistic in our environment Works well within a given Framework Minimal coupling, maximum flexibility. Lacks performance. LCG Application Architecture

  10. For user-centric applications Interactive analysis, configuration, monitoringapplications, etc.

  11. Software Bus • Define a “bus” specifications equivalent to a hardware bus (e.g. VME) where modules implementing different functionality can be plugged easily. Grid Database GUI shell Bus Analysis Tools (Lizard) Experiment Framework (Gaudi) math math LCG Application Architecture

  12. Python as a Software-Bus • Python is an object-oriented scripting language commonly used in a wide variety of domains • But, it could also be seen as framework where you can plug easily “extension modules” in binary form implemented using other languages. • Very easy and natural to interface to C++ classes (C++ API) • Python should only be the “glue” between modules developed in C++ or other languages • The interface (API) for Python extension modules is quite simple and at the same time flexible LCG Application Architecture

  13. Module Communication • Communication between two extension modules is always possible using Python primitive types (a) • Example: I would like to display an histogram of Gaudi • It is more efficient to communicate using a reference to an interface (AIDA?) (b) • The objects in Python are only references Python Python Primitives IH1D* (a) (b) Lizard (C++) Gaudi (C++) Plotter IH1D Both ways are possible!! LCG Application Architecture

  14. Why Python? • Interpreted • However it is quite fast (byte code idea from Java) • Dynamically typed • Not need to declare any variable • Simple syntax • Emphasis by the authors to minimize typing • Shells • Powerful built-in types and modules  Ideal for Scripting and Prototyping LCG Application Architecture

  15. What is Feasible? Very rich set specialized generic modules LHC modules Several GUI toolkits EDG API PVSS XML Database GUI GUI Python shell Gateways to other frameworks JPE rootpython gaudipython math math Java Classes Root Classes Gaudi Framework Very rich set of Python standard modules LCG Application Architecture

  16. Example # Get the ntuple from the ROOT file import rootmodule hfile = TFile('hsimple.root') ntuple = gROOT.FindObject('ntuple') entries = ntuple.GetEntries() nvar = ntuple.GetNvar() tuple = ntuple.GetArgs() # Initialize Excel import win32com.client excel = win32com.client.Dispatch('Excel.Application') wbook = excel.Workbooks.Add() wsheet = wbook.WorkSheets.Add() wsheet.Name = ntuple.GetTitle() # Fill Excel sheet for i in xrange(500) : ntuple.GetEntry(i) for j in range(nvar) : wsheet.Cells(i+1,j+1).value = tuple[j] # Make Excel sheet visible excel.Visible = 1 Filling an Excel spreadsheet from a ROOT ntuple LCG Application Architecture

  17. Ideal Interactive Application Menu GUI Data Browser Canvas (histograms, 3D graphics, etc.) >>> execfile(‘mymacro’) >>> for c in cands : plot(c) >>> Scripting Window LCG Application Architecture

  18. Where are the problems? • The Software-Bus architecture is not applicable everywhere • I would not like to implement the LHCb 2 level trigger with that ! • Software-Bus + Abstract interfaces is a good compromise for user-centric applications (fast prototype) • In addition to “standardize” to the Python extension API we need more • AIDA ?, interoperability between Boost/Swig? … • Mixing several GUI toolkits, thread libraries, etc. may pose serious technical problems. LCG Application Architecture

More Related