Grid Technologies in Disaster Management & Infrastructure Protection

1. Grid Technologies in Disaster Management & Infrastructure Protection Geoffrey FoxComputer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47404 gcf@indiana.edu http://grids.ucs.indiana.edu/ptliupages/presentations/CIGridGDINmar28-04.ppt http://www.infomall.org http://www.grid2002.org

2. Introduction • Grid is NOT a mesh in this talk!!!! • The Grid represents “Internet Scale Distributed Computing” or the “Managed Internet” • Major commercial and research Initiative – IBM HP Oracle Sun SGI Platform • We describe how one can use familiar system of systems (Grid of Grids) language for critical infrastructure and emergency response (command and control) Grids • We give infrastructure examples from flood and earthquake cases with HPC simulations linked to other Grid resources • Discuss GIS and Collaboration Grids

3. e-Business e-Science and the Grid • e-Business captures an emerging view of corporations as dynamic virtual organizations linking employees, customers and stakeholders across the world. • e-Science is the similar vision for scientific research with international participation in large accelerators, satellites or distributed gene analyses. • The Grid or CyberInfrastructure integrates the best of the Web, Agents, traditional enterprise software, high performance computing and Peer-to-peer systems to provide the information technology e-infrastructure for e-moreorlessanything. • A deluge of data of unprecedented and inevitable size must be managed and understood. • People, computers, data and instruments must be linked. • On demand assignment of HPC resources, experts, computers, networks and storage resources

4. e-Defense and e-Crisis • Grids support Command and Control and provide Global Situational Awareness • Link commanders and frontline troops to themselves and to archival and real-time data; link to what-if simulations • Dynamic heterogeneous wired and wireless networks • Security and fault tolerance essential • System of Systems; Grid of Grids • The command and information infrastructure of each ship is a Grid; each fleet is linked together by a Grid; the President is informed by and informs the national defense Grid • Crisis Management and Response enabled by a Grid linking sensors, disaster managers, and first responders with decision support • Grid supports On-demand HPC Simulations • Define and Build DoD relevant Services – Collaboration, Sensors, GIS, Database etc.

5. Each service should beable to run independently on separate machines Database Typical Grid Architecture UserServices Service OrientedArchitectures scale better than traditional Distributed objects Portal Services Re-use Application ServiceLibraries ApplicationCustomization Application Service Application Service Middleware SystemServices SystemServices SystemServices Re-use “Core”Grid Raw (HPC) Resources

6. PortalService Security Catalog A typical Web or Grid Service • In principle, services can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) • The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python PaymentCredit Card Web Services WSDL interfaces Warehouse Shipping control WSDL interfaces Web Services

7. CPUs Clusters Compute Resource Grids MPPs Methods Services Functional Grids Federated Databases Databases Data Resource Grids Sensor Sensor Nets Composing Systems • We are familiar with a hierarchyLines of Code  Methods  Objects  Programs  Packages • Grids extend the software hierarchy to larger distributed systems and link it to a corresponding resource hierarchy Overlay and Compose Grids of Grids

8. R1 R2 Enterprise Grid Dynamic light-weight Peer-to-peer Collaborative Training Grid Students Information Grid Compute Grid Campus Grid Teacher Composing anOverlay Grid

9. Typical Science GridService such as Research Database or simulation Science Grids CI Grids Campus orEnterprise Administrative Grid Transformed by Grid Filterto form suitable for education Education Grid Publisher Grid Learning Management or LMS Grid Student/Parent … Community Grid Digital Library Grid Informal Education (Museum) Grid Inservice Teachers Preservice Teachers School of Education Teacher Educator Grids Education as a Grid of Grids

10. Solid Earth Research Virtual Observatory (SERVO) • NASA funded: JPL, UC Davis, UC Irvine, USC, Brown, Indiana • Web/Grid-services and portlet based Problem Solving Environment (PSE) • Couples data with simulation, pattern recognition software, and visualization • Enable investigators to seamlessly merge multiple data sets and models, and create new queries. • Data • Spaced-based observational data • Ground-based sensor data (GPS, seismicity) • Simulation data • Published/historical fault measurements • Analysis Software • Earthquake fault • Lithospheric modeling • Pattern recognition software • International Version iSERVO • Australia China and Japan as an APEC (Asia-Pacific Economic Cooperation) Initiative – next meeting Beijing July 2004

11. ? GIS Discovery Services Crisis Data RepositoriesFederated Databases Streaming Data Sensors Other InterdependentCI and CrisisGrids Database Database Research Crisis SERVOGrid Data FilterServices Customization Services From Researchto CI/Crisis ResearchSimulations Analysis and VisualizationPortal Cell PhoneMobileNetwork Geoscience Research and Crisis Grids

12. Field Trip Data ? ResearchSimulations GIS Discovery Services RepositoriesFederated Databases Streaming Data Sensors Database Database Research Education SERVOGrid Data FilterServices Customization Services From Researchto Education Analysis and VisualizationPortal EducationGrid Computer Farm Geoscience Research and Education Grids

13. Data Data Filter Filter Filter Data Filter Data OGSA-DAIGrid Services AnalysisControl Visualize Grid Data Filter This Type of Grid integrates with Parallel computing Multiple HPC facilities but only use one at a time Many simultaneous data sources and sinks HPC Simulation Grid Data Assimilation Other Gridand Web Services Distributed Filters massage data For simulation SERVOGrid (Complexity) Computing Model

14. Crisis and CI Grids • We first form Critical Infrastructure CI Grids from the “atomic” functional and resource Grids and Services • This gives water (flood), gas, postal, agricultural etc. Grids • Then we customize, compose and overlay CI with other Grids (such as weather, census data) for • Public Health • Emergency Response (Command and Control) or Crisis Grids • Infrastructure Planning (e.g. transportation planning) • Education (schools) and Training (of managers and first responders) • Interdependency analyses between the different infrastructures • Note need to link to “Cell Phone Grid” for first responders with services that reduce image sizes etc.

15. What are we trying to do • Center for Disaster and Critical Infrastructure Informatics • IUPUI, Indiana, Purdue, openGIS Consortium project to build a generic Crisis/CI Grid infrastructure with a FloodGrid as initial prototype • SERVOGrid (Solid Earth Research Virtual Observatory) building Grid to support Earthquake Forecasting • GlobalMMCS is an open-source Audio-Video conferencing and collaboration system built totally on Web Services and spanning Access Grid, H323, SIP and WebCAM’s. • NaradaBrokering is a Grid message system supporting reliable message delivery, software multicast, firewall friendliness, security. (Grid as managed web) • Hope to partner with NISAC (National Infrastructure Simulation and Analysis Center) on core CIGrids

16. NaradaBrokering Audio/Video Conferencing Client Computer Modem Server Peers NaradaBrokering Broker Network Minicomputer Firewall Laptop computer Workstation Peers Audio/Video Conferencing Client PDA Web Service B Queues Stream Reliable Routedsoftware multi-cast Secure filtered Message delivery Web Service A

17. CI/Crisis Grid View of generic Grid Resource Grid Resource Filter CI/Crisis Grid of Grids • Services in a CI or Crisis Grid fall into three classes • 1) Those that special to Critical Infrastructure or Crises such as biological sensor nets or evacuation planning service • 2) Those that are important but can be taken from other Grids such as GIS or collaboration and security • 3) Those that come from other Grids and are refactored for crises • The simulation is reduced in size • The public health database interface is simplified CI/Crisis Grid

18. Electricity CIGrid Gas Servicesand Filters Flood Servicesand Filters Sensors Visualization openGISServices Compute Collaboration Registry Metadata Data Access/Storage Security Notification Workflow Messaging Physical Network Critical Infrastructure CIGrids Gas CIGrid Flood CIGrid … …

19. openGIS Grid Semantics • Note GIS (Geographical Information System) Grid at heart of all these Grids • Geography Markup Language (GML) is an XML encoding for the specification of the geometry and properties of geographic features. GML utilizes the OpenGIS Abstract Specification geometry model which has been harmonized with the ISO geospatial geometry model. • We are building CI specific ontologies in terms of GML to define faults, satellites etc. • http://ripvanwinkle.ucs.indiana.edu:4780/examples/download/schema/ • Styled Layer Descriptor (SLD) specifies the format of a map-styling language for portraying the output of Web Map Servers, Web Feature Servers and Web Coverage Servers etc. SLD will enable different communities in the Emergency Response area to develop a set of customized portrayal rules that best fit their mission requirements. • This becomes the specification of portals to different composite Grids • Sensor Markup Language (SensorML) defines the information model for discovering, querying and controlling Web-resident sensors. • Observations & Measurements (O&M) defines the information model for observations that are returned from the CrisisGrid sensors.

20. GIS Grid Services I • Web Feature Service (WFS) supports the query and discovery of geographic features delivering GML representations of simple geospatial features in response to queries from HTTP clients. WFS can access geographic features including critical infrastructure features, incident locations, and flood-related geographic features including inundation areas, watershed boundaries, and demographic feature. • Web Coverage Service (WCS) supports the query and discovery of digital geospatial information such as digital elevation models, imagery, orthophotography, weather coverages (such as predicted rainfall, air pressure, wind speed and direction), and any other space-varying flood-related phenomena. • Web Map Service (WMS) uses a SLD portrayal to generate "pictures" of georeferenced feature or coverage data. WMS will provide a means to portray geographic information independent of the underlying data model (WFS or WCS). • Coverage Portrayal Service (CPS) defines a standard interface for producing visual pictures from coverage data typically accessed via WCS with a SLD portrayal.

21. GIS Grid Services II • Web Terrain Service (WTS) augments WMS with advanced visualization including 3D terrains. • Catalog Service - Web Profile (CS-W) is a catalog service that will be built on a general Grid metadata service • Sensor Collection Service (SCS) fetches observations from a sensor or group of sensors and will be integrated with research on Grid sensor services • Sensor Planning Service (SPS) assists in 'collection feasibility plans' and to process collection requests for a sensor or group of sensors. • Web Notification Service (WNS) will be replaced by standard Grid notification service

22. Grid Portals • Portals need to provide services for security, customization, layout, rendering • Jetspeed and GridSphere are two well known portals • Commercial systems like IBM WebSphere similar • These Portals are very important as they encourage “component” model for user interfaces and so this fits service model so every service can be packaged with its (document fragment) user interface • Portlets are good as they support they mixing and matching of services for a particular composed/overlay Grid to allow this to be easily reflected in user interface • New JSR168 portlet Java Standard • NSF Middleware Initiative NMI funding collection of open source Grid portlets

23. The OGCE Computing Grid Portal • Provides Portlets for • Management of user proxy certificates • Remote file Management via Grid FTP • News/Message systems • for collaborations • Grid Event/Logging service • Access to OGSA services • Access to directory services • Specialized Application Factory access • Distributed Workflow • Access to Metadata Index tools • User searchable index • Real Time Collaboration • Audio/Video Conferencing Download from open source http://www.ogce.org

24. Example Capability: File Management • Grid FTP portlet– Allow User to manage remote file spaces • Uses stored proxy for authentication • Upload and download files • Third party file transfer • Request that GridFTP server A send a file to GridFTP server B • Does not involve traffic through portal server User Portal Server Jetspeed 1 of many Portlets GridFTPService GridFTP Server A GridFTP Server B

25. Watershed Entry Page: Select a County FloodGridPortal

26. Watershed Boundaries in Black Open source GIS MapServer presented as a simple portlet

27. Data Download Portal This is where you can download ArcView compatible data files for your watershed. Some of this gets hooked to the runoff model in our connected crisis grid model.

28. GIS Data Archives provide terrain, elevation, and land use data Flood Grid Sensors Grid Workflow Links Data, Sensors, Model Parallel Simulations

29. SERVOGrid HPC Applications • Codes range from simple “rough estimate” codes to parallel, high performance applications. • Disloc: handles multiple arbitrarily dipping dislocations (faults) in an elastic half-space. • Simplex: inverts surface geodetic displacements for fault parameters using simulated annealing downhill residual minimization. • GeoFEST: Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body forces. • VirtualCalifornia: Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space • RDAHMM: Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another. • PARK: Boundary element program to calculate fault slip velocity history based on fault frictional properties; a model for unstable slip on a single earthquake fault. • PDPC: Phase Dynamics Probability Change • Preprocessors, mesh generators • Visualization tools: RIVA, GMT

30. QuakeSim Portal Shots SERVOGrid linking HPC, Data and Visualization SAR Data

31. Run Finley – Simulation from Australia

32. Global-MMCS 2.0 XGSP MCU • We are building an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services. • We will deploy it globally with first release end of May 2004. • The function of A/V media server will be distributed using NaradaBrokering architecture. • Media Servers mix and convert A/V streams • Open XGSP MCU based on the following open source projects • openh323 is basis of H323 Gateway • NIST SIP stack is basis of SIP Gateway • NaradaBrokering is open source messaging from Indiana • Java Media Framework basis of Media Servers

33. Session Server XGSP-based Control Media Servers Filters NaradaBrokering All Messaging Admire SIP H323 Access Grid Native XGSP XGSP Web Service MCU Architecture Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing WebServices NB Scales asdistributed High Performance (RTP)and XML/SOAP and .. Gateways convert to uniform XGSP Messaging NaradaBrokering

34. A/V Collaboration Systems • GlobalMMCS Grid federates existing A/V Protocols • H323 H.323 is defined as an umbrella standard specifying the components to be used within an H.323-based environment. • SIP The Session Initiation Protocol (SIP) defines how to establish, maintain and terminate Internet sessions including multimedia conferences • Access Grid enhanced Mbone A/V tools ( VIC, RAT ) Internet 2 network ( Multicast support )

35. Grid Services in GlobalMMCS I • Audio Mixing The audio mixer creates a mixed audio stream from all the audio streams in the session • Video Mixing Video mixing makes the unicast users watch the pictures of multiple participants in a meeting through one video stream • Video Thumbnail visualize the VS set in the session, embedded into the control panel of each endpoint, which Image grabbers capture video streams and save them as static JPEG files. • All the media processing components can be distributed among the pool of the media servers connected to NaradaBrokering infrastructures. • This generalizes to a HPC farm of “stream servers” doing image processing etc. • Perhaps need 100 node cluster to support 10,000 simultaneous A/V streams

36. H.323, SIP Gateway Servers, A/V Session Server • H.323 and SIP gateway transform their protocol specific messages into XGSP signaling messages so that H.323 and SIP A/V endpoints could communicate with the XGSP A/V session server • The session server implements session management logics • creating/destroying A/V sessions • allowing endpoints to join/leave session • Allowing users to make audio/video selection, managing A/V application components

37. Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms 60 NaradaBrokering-RTP JMF-RTP 50 40 30 Delay (Milliseconds) 20 10 0 0 200 400 600 800 1000 1200 1400 1600 1800 2000 Packet Number

38. Polycom, Access Grid and RealVideo views of multiple streams using GlobalMMCSA/V Web Service

39. Unicast AG Portlet

40. Shared Output Port Collaboration WSDL R U Application orContent source F F WSViewer WSDisplay I I O O Web Service Collaboration as a WSSet up Session with GlobalMMCS defining participants NaradaBrokeringMessage Bus Master WS Viewer WS Display Text Chat Whiteboard GIS Services etc. are AUTOMATICALLYCollaborative if Built as a Grid Services OtherParticipants WS Viewer WSDisplay

41. Integration of PDA, Cell phone and Desktop Grid Access