GRID enabled remote instrumentation and sensors with distributed control

1. EU FP6 Project GRID enabled remote instrumentation and sensors with distributed control Francesco Lelli Istituto Nazionale di Fisica Nucleare – Laboratori di Legnaro Legnaro (PD), Italy

2. The Grid Technologies to extend the limit of a single computer (center) Storage Element Computing Element Grid Gateway Grid Technologies Computing Element User Interface Computing Element

3. Extending the Grid Concepts Grid Gateway Grid Technologies Satellite views to monitor the volcano Terrestrial probes to monitor The volcano activities To model calculations and disaster predictions Control and Monitor Room

4. GridCC Data for Model Calculations Predictions + Instruments Grid Computational Grid GridCC

5. The GRIDCC project: Goals & Objectives • The GRIDCC - Grid Enabled Remote Instrumentation with Distributed Control and Computation – project has the main aim to bring Instruments to the GRID • It is a 3-years EU FP6 project started in September 2004 • Web site: www.gridcc.org • More in particular the project goals are: • Definition of a “Instrument Element” allowing a standard remote access to any type of instrumentation • Tight integration between instrument grid and classical computational grid • Human interaction with Grids via Virtual Control Room (collaborative environment) • Enactment of complex workflows

6. The GRIDCC partners

7. GridCC: The Basic Idea Instrument Element Instrument Element Instrument Element Virtual Control Room Virtual Control Room Computing Element Computing Element Computing Element Storage Elements Storage Elements Storage Element Existing Grid Infrastructures Execution Service

8. Instrument elements (IE) Storage Element (SE) Instrument elements (IE) Storage Element (SE) Instrument Element (IE) Storage Element (SE) Storage Element (SE) Storage Element (SE) Storage Element (SE) Grid Compute element (CE) Compute element (CE) Computing Element (CE) Execution Service WfMS WMS AS GRIDCC Architecture gLite GRID GRIDCC Virtual Control Room Information System (BDII) WMS Work Management System WfMS Work Flow Mng System AS Agreement Service Broker Web Service Interface

9. GridCC PermanentTest Bed VCR, IE WfMS VOMS IE, CE/CREAM,SE, LFC AS, KrbServer WMProxy VCR, SE, IE, BDII

10. GRIDCC main target areas Remote process control Accelerator control (Tele-) Biomedicine Robotics Automotive Electronic microscopes (Large-scale) scientific experiments High energy particle physics (Radio-) Telescopes GRIDCC Middleware Widely Sparse Instrumentation Power Grids Monitoring of the territory Monitoring of the sea Geo-hazard prediction Distributed laboratories Transportation monitoring Sensor network

11. GRIDCC pilot applications PowerGrid Particle Accelerator High Energy Physics CMS Meteorology Device Farm Geohazard Monitoring

12. Storage Element Computing Element Web Services Instrument Element Any Protocol or physical connection Grid Sensor Network Instrument Computing Element Instrument Element W E D F A C B Instrument Instrument Element Requirements 1: Provide a uniform access to the physical devices • 2: Allow a standard grid access to the instruments 3: Allow the cooperation between different instruments that belong to different VOs

13. A new 4 channels element to fit the requirements: The Instrument Element (IE) Custom (JMS, ..... ) Discovery Data/Info Pubblishing I’m here Instrument Element Instrumentation Contorl Web Service GRID Access SRM/GRIDFTP Instruments Access Advance Reservation Custom QoS

14. Access Control Manager Data Flow State Flow Error Flow Monitor Flow Control Flow Instrument Element Architecture IMS The term Instrument Element describes a set of services that provide the needed interface and implementation that enables the remote control and monitoring of physical instruments. IMS Resource Service SRM/SE GridFTP SE Instrument Element Problem Solver VIGS Data Publisher IMS Instrument Manager IMS Proxy Control Manager Data Collector Control Manager Event Processor FSM Engine Instrument Protocols Input Manager Resource Proxy Real Instruments

15. Virtual Control Room (VCR) • The VCR is a web portal • Based on the Gridsphere framework • (portlet technology) and providing • general cooperative tools • event tools • resource tools • communicatin tools • chat • video conf • application specific tools • instrument control • log book • log reports • data monitor

16. VCR examples

17. GRIDCC at Work

18. DAQ IM DAQ RS DAQ IMS FB RB FF xdaq xdaq xdaq Pilot applications: CMS - I • The GridCC middleware has been deployed to control the run of the CMS (Compact Muon Solenoid), one of the four high energy experiments in LHC (Large Hadron Collider) at CERN laboratory. • CMS Magnet Test and Cosmic Challenge (MTCC) I and II, a milestone in the CMS construction, positively carried out. CMSDetector CMS Control Structure User Interface Top IE CSC IE Tracker IE HCAL IE DAQ IE RPC IE Trigger IE DT IE ECAL IE

19. CMS MTCC phase I and phase II carried out in 2006 Scale MTCC versus CMS FEDs: 20 out of 600 3% EVB RUs: 6 out of 600 1% Filter Farm: 14 out of 2000 0.3% Trigger rate: 100 Hz out of 100 kHz 0.1% Event size: 200 kB out of 1 MB 20% IEs 15 out of 150 10 %

20. Maestro Storage IM VCR Main Data Flow (Narval Systems) Resource Service Log Collector Error Collector ….. IM = Function Manager Services Web Services Services Top IM Ancillary IM SlwCtrl IM Narval Systems PSA Chf d’Or Tracking Chf d’Or EVB Chf d’Or SlowCtrl Sub- System Tracking Sub-System PSA Sub-System EVB Sub- System Storage Sub- System Ancillary Sub- System

21. Virtual Control Room Virtual Control Room Pilot applications: Power grid • GRIDCC deployed to monitor: • A 50kW generator • A 1 kw Photo-Voltaic array Instrument Manager Power Grid V.O. Instrument Element Solar Panel ... Gas

22. Elettra Synchrotron Pilot applications: Remote Operation of an Accelerator

23. GRIDCC: other applications • Meteorology (Ensemble Limited Area Forecasting) • Weather forecasting system to detect hazardous weather • Device Farm for the Support of Cooperative Distributed Measurements in Telecommunications and Networking Laboratories • The Device Farm consists of a pool of Measurement Instruments for Telecommunication Experiments • Geo-hazards: Remote Operation of Geophysical Monitoring Network • The monitoring net will be characterized by different levels of activity: stand-by, pre-alert, alert, plus a control modality • An event worth to be monitored is for example the evolution of a “landslip”

24. Custom Board Xilink Virtex IV Grid Custom Electronics FPGA PPC 405 JavaVM Web Service GridCC IM Linux Instrument Element: Scalable on embedded systems 1 Gbps Ethernet IE Instrument Manager Custom Logic Standalone Axis Montavista www.montavista.com USER INTERFACE JamVM http://jamvm.sourceforge.net

25. IE technologies • Web Service compliant (WS-I) • Tomcat + Axis (and Java) and Axis standalone are the main technologies of the IE • All the services are deployed on a single or multiple instances of Tomcat, according to the needs of the application • Message oriented middleware (Pub/Sub) is based on the Java Messaging System (JMS). The following implementations are used in the project • Sun • Narada Brokering • RMM - JMS (GridCC IBM) • MySQL and Oracle are used as Data Base for the RS • SRM interface version 2.2 used

26. Conclusions I • The GRIDCC technology (based on web services) is used to have an homogenous backbone to control and monitor geographically widely distributed instrumentation. • Data taken from the instrumentation can be directly seen by the european computational and storage GRID • CMS experiment is using for its data taken GRIDCC based run control system. The final configuration is target to control O(1000) nodes • AGaTA experiment is building its data taken GRIDCC based run control system. The final configuration is target to control O(1000) nodes

27. Conclusions II • GRIDCC is ending this year having reached all the mail milestones foreseen and having, in particular, a production environment (CMS) daily running and serving hundreds of physicists in commanding and controlling either their sub detectors or the full experiment. • GRIDCC team is now keen to look for new exploitations of their outcomes in different contexts adding, possibly, new developments and customization of  the middleware according to the new needs. • The control and monitor of  pan European  projects like EMSO and KM3NET appears to be very attractive in this sense, as it could be implemented like a  new e-infrastructure across Europe dedicated to the remote control of big scientific apparatus. Moreover the online data integration with the computational and storage GRID provides to fill a gap in the existing computational grid and at the same time give a strong added value to the proposal of this new e-infrastructure.

28. Conclusions III Lambda Network http://sadgw.lnl.infn.it:2002/MapsMonitor Grid Technologies Pervasive Computing Web 2.0 Tools

29. IE IE IE Discovery Instruments (I) • The discovery of the instruments or of the IEs is an issue when the number of elements is high • We can have two cases: • Quasi static cases • The number of IEs are well defined and the single IE is quite complex with a good hardware support • In this case a register based discovery mechanism can be used. • GRIDCC tesbed is using the LCG BDII (Berkely Database Information Index) based on LDAP • The information collected in the BDII follow a GLUE schema and can be used for match making querries Querring Client Querring Client Querring Client BDII Local BDII

30. IE IE IE IE M1 M2 Querring Client Querring Client Querring Client Discovering Instruments (II) • Dynamic cases • The number of IEs can change very quickly, they are very simple devices, often with poor hardware support • The discovery is just use to know which are the online IEs • A new approach has been developed based on Peer to Peer (P2P) protocols • See the demo

31. Question? More: • The GRIDCC Project. Grid enabled Remote Instrumentation with Distribute Control and Computation. Official Webpage of FP6 European funded project. • Video On-Line Demo. A sample video that shows the GRIDCC service orchestration. • Live Demo. A Peer to Peer Approach for the Geo-Location of a Grid of Instruments • GridCC in GRIDToday. Gaetano Maron and Francesco Lelli presented the GRIDCC project in one of the leading online journals. • GridCC in Wikipedia • Instrument Element Facade Online documentation for Instrument Elements developers. Thx for your time!