GridCC: Real-time Instrumentations Grids

1. GridCC: Real-time Instrumentations Grids A real-time interactive GRID to integrate instruments, computational and information resources widely spread on a fast WANAndrea Petrucci Istituto Nazionale di Fisica NucleareLaboratori Nazionali di Legnaro, Legnaro Italy

2. Overview • The GridCC Project: Introduction • Bringing Instrument into the Grid: the Instrument Element • The GridCC Test-bed: Pilot application • Instrument Instrumentation • Fast Instrument Communication Channel • Standard Grid Interaction

3. It is a 3 years project. Started the 1st September 2004 Funded by EU in the Frame Program 6 10 Partners from 3 EU Countries + (Israel) About 40 people engagged www.gridcc.org General on the GridCC Project

4. The Grid Technologies extend the limit of a single computer (center) Storage Element Computing Element Grid Gateway Grid Technologies Computing Element User Interface • Current Grid technologies offer unlimited computational power and storage capacity for scientific research and business activities • However concrete use cases demand more close interaction between various types of instruments accessible from the Grid elements

5. Extending the Grid Concepts Data for Model Calculations Predictions + Computational Grid Instruments Grid GridCC Project • The goal is to build a widely distributed system that is able to remotely controls and monitors complex instruments

6. Instrument Element: global scenario Instrument Element Instrument Element Instrument Element Virtual Control Room Virtual Control Room Computing Element Computing Element Computing Element Storage Elements Storage Elements Storage Element Exec. Service WfMS WMS Web Service Interface AgrS Existing Grid Infrastructures User direct Action Indirect Action

7. Direct access to IE SE (and CE) possible but often not desirable Storage Element (SE) Storage Element (SE) Storage Element (SE) Compute element (CE) Compute element (CE) Compute element (CE) The GridCC Architecture Virtual Control Room (VCR) All end user access is via the VCR Virtual Control Room (VCR) Collaborative Services (CS) The IE is a virtualization of the real physical instrument Users generally not working alone Information and Monitoring Services (IMS) “Fast” all pervasive messaging system Instrument elements (IE) Execution Services Information System (IS) Instrument elements (IE) Slowly updating information Compute and Storage Elements (with advanced reservation) Of course there may be many IEs Instrument elements (IE) Security is essential to the success of the project Of course Many CEs and SEs Watching (via the IMS) for problems anywhere in the system and acting to resolve them. More complex workflows, including advanced reservation and QoS guarantees , allowed Security Services Global Problem Solver

8. Storage Element Computing Element Web Services Instrument Element W Any Protocol or physical connection Grid Sensor Network Instrument E D Computing Element Instrument Element F A C B Instrument IE Requirements 1: Provide a uniform access to the physical device • 2: Allow a standard grid access to the instruments 3: Allow the cooperation between different instruments that belong to different VOs

9. Instrument Element: a Black Box Fast communication channel IE Grid Interaction Instrument Data Mover VIGS Instrumentation • Quick Answers to the previous slide: • The VIGS provide the a uniform instrument instrumentation way • The fast communication channel disseminate the acquired information between instruments • The Data Mover provide a standard Grid Interface in order to be accessed by others Grids components like the SE and the CE Instruments • The term Instrument Element describes a set of services that provide the needed • interface and implementation that enables the remote control and monitoring of physical instruments.

10. Access Control Manager Data Flow State Flow Error Flow Monitor Flow Control Flow Instrument Element Architecture Virtual Instrument Grid Service (VIGS) • The IE entry point is constitued by a Web Service interface called Virtual Instrument Grid Service(VIGS) • The VIGS allows users to access and control the specific physical Instrument from any part in the world using a VCR. Instrument Element create() Inf & Mon Service Resource Service Problem Solver destroy() execute() Data Mover getState() Instrument Manager IMS Proxy Control Manager Data Collector Control Manager Event Processor FSM Engine Input Manager Resource Proxy Real Instruments

11. Instrument Element Inf & Mon Service Resource Service Problem Solver Access Control Manager Data Mover Instrument Manager Instrument Element Implementations • The IE components are typically implemented into a fully equipped Machines (e.g. dual core cpus, large memory, large disks, etc). This is true for RS, IMS and PS. For IM (and DM) there are 2 possibilities, according to the application type: • IM implemented in a fully equipped machine • IM embedded into the instrument that should be controlled IMS RS IM IM IM IM Embedded Web Service

12. Instruments Instrument Manager Control Manager IMS Proxy Event Processor FSM Engine Data Collector Input Manager Resource Proxy Instrument Manager Control Flow Plug-in modules to interface to the instruments Customizable Data Flow Monitor Flow State Flow Error Flow • IM is composed by 3 main components: • Control Manager: • Input Manager : It handles all the input events of the IM. These includes commands from GUIs or other IMs,errors/state/log/monitor messages. • Event Processor: It handles all the incoming message and decide where to send them. It has processing capability • FSM: A finite state machine is implemented. • Resource Proxy: It handles all the outgoing connections with the resources. • Data Collector: It get data from the controlled instruments and make them available to the data mover. A local storage of the data is even foreseen. • IMS Proxy: It receives error/state/log/monitor information from the controlled resources and forward them to IMS.

13. Resource Service Architecture Partition/Configuration retrieve methods • The Resource Service (RS) handles all the resources of an IE and manages their partition (if any). • A resource can be any hardware or software component involved in the IE (instruments, Instrument Managers, IMS components) • RS stores the configuration data of the resources and download them to resource target when necessary • Resources can be discovered, allocated and queried. • It is the responsibility of the RS to check resource availability and contention with other active partitions when a resource is allocated for use. • A periodic scan of the registered resources keeps the configuration database up to date. • RS is interfaced to the WMS Discovery Manager Available Resources Partition and Lock setting methods Subscribe Manager Partition Definitions RS Data Bases Configuration setting methods Partition&Lock Manager Configuration Definitions Discovery methods Configuration Manager

14. Instruments Instruments Instruments Instrument Manager Instrument Manager Instrument Manager Information and Monitor System (IMS) • The Information and Monitor Service (IMS) collects messages and monitor data coming from GRID resources and supporting services and stores them in a database. There are several types of messages collected from the sub-systems. The messages are catalogued according to their type, severity level and timestamp. Data can be provided in numeric formats, histograms, tables and other forms. • The IMS collects and organizes the incoming information in a database and publishes it to subscribers. These subscribers can register for specific messages categorized by a number of selection criteria, such as timestamp, information source and severity level. Errors Log info Monitor State PUBLISHERS (Instruments nodes) SUBSCRIBERS

15. Instrument Manager Instrument Manager Instrument Manager Instrument Manager IMS Proxy IMS Proxy IMS Proxy IMS Proxy Control Manager Control Manager Control Manager Control Manager Problem Solver Step 3 On-line information can be analyzed in order to detect possible malfunctions Step 1 The control manager can perform an autonomous recovery action where the cost for the determination it is not so heavy . Problem Solver Pub/Sub On Line Analisys DB Data Mining Tools State Flow Error Flow Monitor Flow Algorithms evaluations : Rule Induction, Tree, Functions, Lazy, Clusters and Associative Step 2 Persistent information can be analyzed in order to extract knowledge

16. Data Collector Data Collector Data Collector IM IM IM Standard Grid Interaction: Data Mover IE • The task of this element is to get data from the “data collector” of the IM • Data can be accessed via: • Web service interface for generic data dump (e.g. slow storage, spy stream, etc.) • grid storage element (SE) and available CEs can access to the data via an SRM Interface • Http server and TCP communication for high performance had-hoc data transfer • The Data Mover exposes its methods to the IE web service and can be instrumented itself as an instrument. Web Service Interface: get_data() Data Mover SRM interface Instrument Resources Http Server and TCP/IP raw socket

17. Main GRIDCC Pilot Applications CMS detector - CERN Power Grid - Brunel Intrusion Detection System – NTUA/IASA Far remote control of accelerator - Elettra

18. Main GridCC Pilot Applications: • The Run Control prototype is controlling and monitoring a Data Acquisition (DAQ) system of the High Energey Physical Experiment CMS at the CERN Large Handron Collider.

19. O(104 ) distributed Objects to control configure monitor On-line diagnostics and problem solving capability Highly interactive system (human reaction time - fraction of second) World Wide distributed monitor and control The CMS Data Acquisition 40 MHz 2 107 electronics channels 100 Hz

20. DAQ Trigger TTS FilterFarm FedBuilder RuBuilder CMS Prototype: IEs at work CMS Instrument Elements • GridCC middleware used for CMS MTCC (Magnet Test and Cosmic Challenge) • - 11 Instrument Elements with a • hierarchical topology • - Instruments are in these case Linux • hosts where the cms on-line software • is running • - More than 100 controlled hosts TOP Det 1 Det 1 Det 1 Detector 8 GTPe 1 DAQ DAQ IE Instrument Managers

21. Power Grid V.O Virtual Control Room Virtual Control Room Solar ... Gas Power Grid • The Power Grid prototype is controlling and monitoring power generators (Solar, gas ..) Instrument Manager Instrument Element

22. Remote Operation of an Accelerator Elettra Synchrotron • Far remote operation of an accelerator facility in­volves the planning of accelerator operations, the maintenance of the accelerator and its trouble­shooting, the repair of delicate equipment, understanding and pushing performance limitations, performing studies, performing commissioning and set ups and rou­tine operations.

23. The other GridCC pilot applications • Meteorology (Ensemble Limited Area Forecasting) • Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories • Geo-hazards: Remote Operation of Geophysical Monitoring Network (see first slides) • Medical Devices need a close loop between the data acquisition and the output result

24. Conclusion • The GridCC project is integrating instrument into traditional computational/storage Grids. • IEs need an high interaction and interactivity between itself and the users. • The GridCC IE implementation is currently installed in heterogeneous applications

25. Question..... • Thank you for your time More information: www.gridcc.org Acknowledgement: The GridCC project is supported under EU FP6 contract 511382.