1 / 31

Outline

GRIDCC A realtime interactive GRID to integrate instruments, computational and information resources widely spread on a fast WAN Gaetano Maron Istituto Nazionale di Fisica Nucleare Laboratori Nazionali di Legnaro Legnaro Italy. Outline. Brief introduction to the project Main Pilot Applications

phiala
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRIDCCA realtime interactive GRID to integrate instruments, computational and information resources widely spread on a fast WANGaetano MaronIstituto Nazionale di Fisica NucleareLaboratori Nazionali di LegnaroLegnaroItaly

  2. Outline • Brief introduction to the project • Main Pilot Applications • The GRIDCC Services • Technologies and performances • Conclusions

  3. GRIDCC main goals • ... the GRIDCC project extends the state of the art of computing Grid technologies, by introducing the handling of real-time constraints and interactive response into the existing Grid middleware • Our goal is to build a widely distributed system that is able to remotely control and monitor complex instrumentation …These new applications introduce requirements for real-time and highly interactive operation of GRID resources. • One of the main objectives of the project is to verify the feasibility of a Grid-based remote control of systems requiring real-time response with real applications running on existing Grid test beds over both national and international network infrastructures (e.g. GEANT). • GRIDCC integrates a “grid of instrumentation” into existing Grid infrastructures that provide the computational power and storage needed for the applications ….

  4. Partecipants

  5. The Origin of the ProjectThe control of CMS Experiment

  6. O(104 ) distributed Objects to control configure monitor On-line diagnostics and problem solving capability Highly interactive system (human reaction time - fraction of second) World Wide distributed monitor and control The CMS Data Acquisition 2 107 electronics channels 40 MHz 100 Hz

  7. From the CMS Control and Monitor System .... Supporting Services Virtual Control Room Diagnostic Tools Standard comm. protocols Interface to the “Instrumentation” “Instrumentation”

  8. .... to the GRIDCC project Use of the Grid technology, as extension of the Web Service Technologies, to develop a a widley distributed control system with access to grid enabled computing and data storage facilities Processing Farm Virtual Cntr. Room Instrument 1 Processing Farm Supporting Services Instrument 2 Data Storage Diagnostics Instrument 3 Virtual Cntr. Room

  9. GRIDCC Layout Instr. Instr. Instr. Instr. Instr. Instr. Instr. Instr. Farm Instr. Tele prsnce DBs WP4: RT Access To existing Grid services Existing GRID facilities WP3: Grid Enable Instrumentation Virtual Instrument Grid Services Farm Services Storage Services Imperial INFN User Interface Diagnostic Service Virtual Control Room Test Bed Grid Infrastructure (Web-Service Infr.) Problem Solver Service WP2: Real-time and Interactive Web Services User Interface WP5: Cooperative Environment Data Mining Tool Virtual Control Room User Interface BRUNEL Knowledge based Services WP3: Grid Enable Instrumentation Elettra Video Conf. & Chat Service INFN Security & login Service Information Service (Monitor) Work Flow Engine Service Job Control Resource Service Supporting Services Cooperative Environment

  10. Application Fields • Experimental Sciences • Take control of a experiment from a distance (remote operation and control, data taking and data analysis): • High Energy, Nuclear and Solid State Physics • Electronic Microscopes • Telescopes • Monitoring and analysis of the territory (e.g. disaster analysis) • Meteorology • Geophysics • Bio-medics • Integration of remote operation, data taking, data analysis and data storage of sophisticated instruments like: • Mammography • Pet, TAC, NMR etc. • Industrial Applications • widely distributed controls • Electrical power grid • Public transportation • ……

  11. Pilot Application IPower Grid • In electrical utility networks (or power grids), the introduction of very large numbers of ‘embedded’ power generators often using renewable energy sources, creates a severe challenge for utility companies. • Existing computer software technology for monitoring and control is not scalable and cannot provide a solution for the many thousands of generators that are anticipated. • GridCC technology would allow the generators to participate in a VO, and consequently to be monitored and scheduled in a cost-effective manner • Embedded power generator is still in developing phase. GridCC project has access to its full computer emulation. So the Power Grid application will consist of a network (O(100)) of emulated embedded power generators and their full control and monitor operation

  12. Pilot Application II (Far) Remote Operation of Accelerator Facility • Far remote operation of an accelerator facility (i.e. the Elettra Control Room in Italy) involves the planning of accelerator operations, the maintenance of the accelerator and its trouble­shooting, the repair of delicate equipment, understanding and pushing performance limitations, performing studies, performing commissioning and set ups and routine operations. • All these activities are based on large amounts of information, which are at present accessible only at the accelerator site. Remote control of an accelerator facility has the potential of revolutionising the mode of operation and the degree of exploitation of large experimental physics facilities. • This pilot application will combine elements of immersive (i.e. providing the feeling to be present at the remote location) communication and cooperation technology. This includes video and audio presence, allowing the simultaneous operation of the same instruments, having access to the same accelerator controls and the relevant data, meeting easily and spontaneously and providing full awareness of the presence of the collaborators.

  13. Pilot Application IIIControl and monitor of high energy experiments • This application involves the use of the Grid in a real-time environment to control and monitor remote large-scale detectors. • This application will make use of a High-Energy Physics (HEP) experiment, the CMS detector which is currently under construction at the future LHC collider at CERN. Data taking is foreseen by 2007, but several pre-production activities are planed. • (See previous slides for some more details about CMS detector. ) • This application will be developed along the CMS on-line software developing and will have same time schedule and delivery terms.

  14. The other GridCC pilot applications • Meteorology (Ensemble Limited Area Forecasting) • Analysis of neuro-physiological data (migraine attacks treatments) • Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories • Geo-hazards: Remote Operation of Geophysical Monitoring Network

  15. The GRIDCC Services • Supporting Services • Security Service • login and user account management; security issues • Resource Service (RS) • GRIDCC resources (including instrumentation controller nodes) handling and their partitioning; GRIDCC resources configuration • InformartionAnd Monitor Service (IMS) • Collectesmessages and monitor data from the GRIDCC resources; distributes them to the subscribers • Job Control • Starts, monitors and stops the software elements of GRIDCC, including the Instrument components • Problem Solver • Uses information from the RS and IMS to identify mulfunctions and attempts to provide automatic recovery procedures where applicable • Virtual Instrument Controllers (VIGS) • Instrument controllers, hierarchy of controllers • Transform requests from the UI to proper actions to be sent to the instrumentation

  16. General Requirements for the GridCC Services • About 104 nodes/instruments to be controlled and monitored (for the more demanding application) • The nodes/instrument are controlled by VIGSs. • Round trip time to reach all the nodes must be in the order of human reaction time. A hierarchy of VIGS allows to reduce such time. • Concurrent partitions should be possible (Resource Service) • Information collection from all the node to reach a “single point” of storage. Collection time fast enough to allow monitoring , error detection , alarms, etc. Aggregate throughput in the order of 104 message/s (IMS) • On-line diagnostics and problem solving fast enough to be useful (from seconds to minutes) (Problem Solver) • Real time requirement • This requirement affects both network and Web Service QoS definitions including parameter for : • Delivery certezza • Response to a request in a give amount of time

  17. The GRIDCC ServicesResource Service V I G S V I G S V I G S Instruments Instruments Instruments Instruments description • The Resource Service (RS) handles all the GRID resources and manages their partition (if any). • A resource can be any hardware or software component involved in the GRID. • Resources can be discovered, allocated and queried. • Partitions can only use available resources. • It is the responsibility of the RS to check resource availability and contention with other active partitions when a resource is allocated for use. • A periodic scan of the registered resources will keep the configuration database up to date. Instrument configuration DAQ description

  18. Information and Monitor System V I G S V I G S V I G S Instruments Instruments Instruments • The Information and Monitor Service (IMS) collects messages and monitor data coming from GRID resources and supporting services and stores them in a database. There are several types of messages collected from the sub-systems. The messages are catalogued according to their type, severity level and timestamp. Data can be provided in numeric formats, histograms, tables and other forms. • The IMS collects and organizes the incoming information in a database and publishes it to subscribers. These subscribers can register for specific messages categorized by a number of selection criteria, such as timestamp, information source and severity level. Errors Log info Monitor State PUBLISHERS (Instruments nodes) SUBSCRIBERS

  19. Problem Solver Step 1 Step 2 Step 3 This Service identifies malfunctions of the GRICC system and determines possible recovery procedures. It subscribes to the IMS to receive theinformation it is interested in. The information is processed by a correlation engine and the result is used to determine a potential automatic recovery action, or to inform the user providing any analysis results it may have obtained.

  20. Virtual Instrument Grid Service VIGS IMS VIGS is a set of services that enables the remote control, monitoring and overall operation, via GRID protocols, of a set of real instruments Virtual Instrument Grid Service Errors, Log Info, Monitor, State Control Gateway InfoServ Proxy Instrument Manager Data Mover Instrument Manager Controls, Status To Grid Farms, Data storage, Visualization, etc. Virtual Control Room Real Instruments or Set of Instruments

  21. Hierarchy of VIGSs VIGS VIGS VIGS VIGS VIGS VIGS VIGS VIGS Virtual Control Room IMS CE/SE VIGS Control Flow Errors Flow Data Flow Real Instruments

  22. Project Timing Years 3 1 2

  23. Project Time Schedule

  24. Preliminary Service Prototypes • Resource Service, IMS and a reduced version of VIGS exist as preliminary study prototypes. The aim is to gain experience with the technologies and provide a preliminary test bed for our appliccations. • The following technologies have been used: • Tomcat based (Java servlet container) • SOAP/XML (Jaxm) • Castor • MySQL and Oracle DB, JDBC • Sun Message Queue (JMS) for IMS • An integrated version of the above mentioned services and VIGS is now in operation to control a 128 nodes (instruments) system

  25. Performance Issues of the prototype: Round Trip Time VIGS Soap/XML VIGS VIGS Soap/XML

  26. Performance Issues of the prototype: Info Pub/Sub Performance Msg BROKER Msg Filter SUBSCRIBERS PUBLISHERS Persistency M.Q. JMS Based Broker JMS msg Tomcat Based Broker SOAP/XML msg Dual Xeon 1.8 GHz

  27. Comments on the performance requirements and guess for the GridCC technologies • Single VIGS routing capability should be in the order of 102 msg/s. Due to the topology of the application (hierarchical) this number is a reasonable compromise. The prototype (Tomcat + Soap) fits with this number. Web Service based VIGS should fit easily with this figure. • IMS message broker capability. Due to the nature of the messages collected by this service (asynchronous error messages, state changes, monitor information, etc.) we require at least 1000 msg/s per broker. The prototype shows some limitation with Tomcat + Soap scheme. JMS approach behaves properly.

  28. Web Service Performances • Sun J2EE AS • Dual Xeon 1.8 GHz • 10 tag XML doc • remote method invocation • only ack as answer • Glue Web Service • Dual Xeon 1.8 GHz • 10 tag XML doc • remote method invocation • only ack as answer

  29. Grid Technologies for the project • Specifications we are looking at: • WS Agreement • to define the QoS affecting the real time behaviour (as defined at the beginning) of the web service • WS Resource Framework • State full web service • WS Addressing • WS-Notification • WS-Federation

  30. Technology review in progress • Web/Grid Service • WS-I and/or WS-I+ based platforms • OMII • Java Sun ? • Web Sphere ? • JBoss ? • IBM emerging toolkit • WS-RF, WS-ResourceProp, WS-ResourceLifetime, WS-Notification, WS-ServiceGroup, WS-BaeFaults • Globus Toolkit 4 • EGEE gLite • Message pub/sub systems • NaradaBrokering • Sun Messages Queue 3.5 • WebSphere MQ Series

  31. Conclusions

More Related