Sensor Web Enablement (SWE) and SensorML August 2008

1. Sensor Web Enablement (SWE)and SensorMLAugust 2008 Mike Botts mike.botts@uah.edu VAST Team - NSSTC University of Alabama in Huntsville

2. The “Big Hammer” Approach to Sensor Fusion (circa 1991)

3. Importance of Standards

4. Benefits and Challenges of Standards • Ideally the benefits of standards are that they: • Enable interoperability of data and information • Enable development and thus availability of tools (commercial and open-source) for meeting a variety of needs • Enable supply of commodity hardware/software from a variety of vendors • Enable rapid deployment … plug-and-play • Allow one to focus on the real job and not the technology • Some possible challenges with standards are: • Standard may not exist when you need it and developing them is too slow for your needs • Buying into a standard too early can result in • Choosing a losing standard (e.g. Betamax, HD-DVD) • Investment of effort that you may not be able to afford • Tools don’t exist to realize immediate benefits • Some successful standards • WWW (html, httpd, TCP-IP, etc) • USB, • Microsoft Windows, Linux, and MacOS • jpeg, mpeg • The best standards are those we don’t need to think about anymore

5. Standards Buy-In Decision Chart Standards-based Do-it-yourself Relative return on effort • Do-it-yourself may result in faster but limited return • Return on standards tends toincrease with time sweet-spot implementer early implementer • Early implementers may see delayed return on investment • Sweet-spot implementers may reap maximum benefit for least effort Time

6. OpenGeospatial Consortium (OGC)

7. Open Geospatial Consortium • The Open Geospatial Consortium, Inc (OGC) is an international industry consortium of 334+ companies, government agencies and universities participating in a consensus process to develop publicly available interface specifications and encodings. • Open Standards development by consensus process • Interoperability Programs provide end-to-end implementation and testing before spec approval • Standard encodings (e.g. GML, SensorML, O&M, etc.) • Geography Markup Language (GML) – Version 3.2 • Style Layer Description language (SLD) • SensorML • Observations and Measurement (O&M) • Standard Web Service interfaces; e.g.: • Web Map Service (WMS) • Web Feature Service (WFS) • Web Coverage Service (WCS) • Catalog Service • Open Location Services – used by communications and navigation industry • Sensor Web Enablement Services (SOS, SAS, SPS)

8. Sensor Web Enablement

10. Basic Desires • Quicklydiscover sensors and sensor data (secure or public) that can meet my needs – location, observables, quality, ability to task • Obtain sensor information in a standard encoding that is understandable by me and my software • Readily access sensor observations in a common manner, and in a form specific to my needs • Task sensors, when possible, to meet my specific needs • Subscribe to and receive alerts when a sensor measures a particular phenomenon

11. Heterogeneous sensor network Airborne Satellite Decision Support Tools In-Situ monitors Bio/Chem/Rad Detectors Surveillance • sparse • disparate • mobile/in-situ • extensible Sensor Web Enablement • discovery • access • tasking • alert notification web services and encodings based on Open Standards (OGC, ISO, OASIS, IEEE) Models and Simulations • nested • national, regional, urban • adaptable • data assimilation • vendor neutral • extensive - flexible • adaptable Sensor Web Enablement Framework M. Botts -2004

12. Background -1- OGC Web Services Testbed 1.2: • Sponsors: EPA, General Dynamics, NASA, NIMA • Specs: SOS, O&M, SensorML, SPS, WNS • Demo: Terrorist, Hazardous Spill and Tornado • Sensors: weather stations, wind profiler, video, UAV, stream gauges • Specs advanced through independent R&D efforts in Germany, Australia, Canada and US • Sensor Web Work Group established • Specs: SOS, O&M, SensorML, SPS, WNS, SAS • Sensors: wide variety OGC Web Services Testbed 1.1: • Sponsors: EPA, NASA, NIMA • Specs: SOS, O&M, SensorML • Demo: NYC Terrorist • Sensors: weather stations, water quality SensorML initiated at University of Alabama in Huntsville: NASA AIST funding 1999 - 2000 2001 2002 2003-2004

13. Background -2- OGC Web Services Testbed 3.0: • Sponsors: NGA, ORNL, LMCO, BAE • Specs: SOS, O&M, SensorML, SPS, TransducerML • Demo: Forest Fire in Western US • Sensors: weather stations, wind profiler, video, UAV, satellite SAS Interoperabilty Experiment OGC Web Services Testbed 4.0: • Sponsors: NGA, NASA, ORNL, LMCO • Specs: SOS, O&M, SensorML, SPS, TransducerML, SAS • Demo: Emergency Hospital • Sensors: weather stations, wind profiler, video, UAV, satellite OGC Web Services Testbed 5.1 • Sponsors: NGA, NASA, • Specs: SOS,SensorML • Demo: Streaming JPIP of Georeferenceable Imagery; Geoporocessing Workflow • Sensors: Satellite and airborne imagery SWE Specifications toward approval: SensorML – V0.0 TransducerML – V0.0 SOS – V0.0 SPS – V0.0 O&M – Best Practices SAS – Best Practices 2006 2005 2007

14. Brief Video Interlude • Discovery of SWE services using registry • Access and portrayal of SWE Observation data

15. SWE Specifications • Information Models and Schema • Sensor Model Language (SensorML) for In-situ and Remote Sensors - Core models and schema for observation processes: support for sensor components, georegistration, response models, post measurement processing • Observations and Measurements (O&M) – Core models and schema for observations • TransducerML – adds system integration and multiplex streaming clusters of observations • Web Services • Sensor Observation Service - Access Observations for a sensor or sensor constellation, and optionally, the associated sensor and platform data • Sensor Alert Service – Subscribe to alerts based upon sensor observations • Sensor Planning Service – Request collection feasibility and task sensor system for desired observations • Web Notification Service –Manage message dialogue between client and Web service(s) for long duration (asynchronous) processes • Sensor Registries – Discover sensors and sensor observations

16. Status • Current specs are in various stages • SensorML (and SWE Common) – Version 1.0.1 • TransducerML – Version 1.0 • Observations & Measurement – Version 1.0 • WNS – Request for Comments • SOS – Version 1.0 • SPS – Version 1.0 • SAS – Being released for vote • Approved SWE standards can be downloaded: • Specification Documents: http://www.opengeospatial.org/standards • Specification Schema: http://schemas.opengis.net/

17. SensorML

18. What is SensorML? • XML encoding for describing sensor processes • Including sensor tasking, measurement, and post-processing of observations • Detectors, actuators, sensors, etc. are modeled as physical processes • Open Standard – • Approved by Open Geospatial Consortium in 2007 • Supported by Open Source software (COTS development ongoing) • Not just a metadata language • enables on-demand execution of algorithms • Describes • Sensor Systems • Processing algorithms and workflows

19. Why is SensorML Important? • Importance: • Discovery of sensors and processes / plug-n-play sensors – SensorML is the means by which sensors and processes make themselves and their capabilities known; describes inputs, outputs and taskable parameters • Observation lineage – SensorML provides history of measurement and processing of observations; supports quality knowledge of observations • On-demand processing – SensorML supports on-demand derivation of higher-level information (e.g. geolocation or products) without a priori knowledge of the sensor system • Intelligent, autonomous sensor network – SensorML enables the development of taskable, adaptable sensor networks, and enables higher-level problem solving anticipated from the Semantic Web

20. Non-Physical Processes Physical Processes Atomic Processes Composite Processes SensorML Processes Processes where physical location or physical interface of the process is not important (e.g. a fast-Fourier process) Processes where physical location or physical interface of the process is important (e.g. a sensor system) Processes that are considered Indivisible either by design or necessity Processes that are composed of other processes connected in some logical manner

21. Example Atomic Processes • Transducers (detectors, actuators, samplers, etc.) • Spatial transforms (static and dynamic) • Vector, matrix, quaternion operators • “Sensor models” • scanners, frame cameras, SAR • polynomial models (e.g. RPC, RSM) • tie point model • Orbital models • Geospatial transformations (Map projection, datum, coordinate system) • Digital Signal Processing / image processing modules • Decimators, interpolators, synchronizers, etc. • Data readers, writers, and access services • Derivable Information (e.g. wind chill) • Human analysts

22. Example Composite Processes • Sensor Systems, Platforms • Observation lineage • from tasking to measurement to processing to analysis • Executable on-demand process chains: • geolocation and orthorectification • algorithms for higher-level products • e.g. fire recognition, flood water classification, etc. • Image processing, digital signal processing • Uploadable command instructions or executable processes

23. Status of SensorML and SWE Common • SensorML history • Influenced by interoperability challenges for satellite sensors at NASA • Started at UAH in 1998 under NASA AIST funding; brought into OGC in 2000 • Approved as Public Discussion Paper (2002) • Approved as Recommended Paper (2004) • OGC 05-086 approved as Best Practices Document in Bonn (Nov 2005) • OGC 05-086r3 approved as Version 0.0 Technical Specification in July 2006 • OGC 07-000 approved as Technical Specification Version 1.0 on June 23, 2007 • Current: document (OGC 07-000) • Approved Version 1.0 of SensorML and SWE Common data types • Official document available at OGC ( http://www.opengeospatial.com ) • Official Reference Schema resides online at http://schemas.opengis.net/ • Doc and schema also available at http://vast.uah.edu/SensorML

24. Where and how can SensorML processesbe used?

25. SensorML provides metadata suitable for discoveryof sensors and processes Find all remote sensor systems measuring in the visible spectral range with ground resolution less than 20m.

26. Discovery Based on SensorML Credit: Northrop Grumman PulseNet Project

27. Specific Discovery Needs • Unique resource ids used throughout SWE; • sensor centric example: • Find sensors that can do what I need (returns id=“urn:ogc:id:sensor:123”) • Get me a full description of this sensor urn:ogc:id:sensor:123 • Now, find a service (SPS) that lets me task this sensor urn:ogc:id:sensor:123 • Find all services (SOS) where I can get observations from this sensor urn:ogc:id:sensor:123 • Find all processes that can be applied to this sensor output to generate the information I require • Catalog profiles for each need: • SPS, SOS, SAS services • sensors and processes • observations • terms (either through dictionaries or ontologies)

28. Need for Term Definitions used in SensorML and SWE • Observable properties / phenomena / deriveable properties (“urn:ogc:def:property:*) • temperature, radiance, species , exceedingOfThreshold, earthquake, etc. • rotation angles, spectral curve, histogram, etc. • Capabilities, Characteristics, Interfaces, etc. (“urn:ogc:def:property:*”) • Width, height, material composition, etc. • Ground resolution, dynamic range, peak wavelength, etc. • RS-232, USB-2, bitSize, baud rate, base64, etc. • Sensor and process terms (“urn:ogc:def:property:*”) • IFOV, focal length, slant angle, etc. • Polynomial coefficients, matrix, image, etc. • QA/QC terms and tests • Identifiers and classifiers (“urn:ogc:def:identifierType:*; urn:ogc:def:identifier:*” ) • Identifiers – longName, shortName, model number, serial number, wingID, missionID, etc. • Classifiers – sensorType, intendedApplication, processType, etc. • Role types (“urn:ogc:def:role:*”) • Expert, manufacturer, integrator, etc. • Specification document, productImage, algorithm, etc. • Sensor and process events (“urn:ogc:def:classifier:eventType:*”) • Deployment, decommissioning, calibration, etc.

29. SensorML Observation Supports description of Lineage for an Observation Within an Observation, SensorML can describe how that Observation came to be using the “procedure” property

30. SensorML Observation On-demand processing of sensor data SensorML processes can be executed on-demand to generate Observations from low-level sensor data (without a priori knowledge of sensor system)

31. Sensor 1Scanner Sensor 2IMU Sensor 3GPS SensorML – Sensor Systems System - Aircraft IR radiation Digital Numbers Pitch, Roll, Yaw Tuples Attitude Lat, Lon, Alt Tuples Location Mike Botts, Alexandre Robin, Tony Cook - 2005

32. AIRDAS UAV Geolocation Process Chain Demo AIRDAS data stream geolocated using SensorML-defined process chain(software has no a priori knowledge ofsensor system) AIRDAS data stream (consisting of navigation data and 4-band thermal-IR scan-line data)

33. SensorML Observation Observation On-demand processing of higher-level products SensorML processes can be executed on-demand to generate higher-level Observations from low-level Observations (e.g. discoverable georeferencing algorithms or classification algorithms)

34. On-demand Geolocation using SensorML AMSR-E SSM/I TMI & MODIS footprints MAS TMI Geolocation of satellite and airborne sensors using SensorML Cloudsat LIS

35. SensorML Clients can discover, download, and execute SensorML process chains SensorML-enabled Client (e.g. STT) SLD OpenGL SOS Stylers For example, Space Time Toolkit is designed around a SensorML front-end and a Styler back-end that renders graphics to the screen

36. Incorporation of SensorML into Space Time Toolkit Space Time Toolkit being retooled to be SensorML process chain executor + stylers

37. SensorML-Enabled Web Services

38. Observation SensorML can support generation of Observations within a Sensor Observation Service (SOS) SOS Web Service SensorML request For example, SensorML has been used to support on-demand generation of nadir tracks and footprints for satellite and airborne sensors within SOS web services

39. Incorporation of SensorML into Web Services SensorML process chains have been used to drive on-demand data within services (e.g. satellite nadir tracks, sensor footprints, coincident search output)

40. SensorML can support tasking of sensors within a Sensor Planning Service (SPS) SPS Web Service SensorML request For example, SensorML will be used to support tasking of video cam (pan, tilt, zoom) based on location of target (lat, lon, alt)

41. SPS control of Web Cam

42. Dual VASTCAMS (1 & 2)

43. 3D Reconstruction of Storms using SensorML Multi-directional views 3D Reconstruction of storms

44. SensorML for Portrayal

45. SWE Visualization Clients can render graphics to screen SensorML-enabled Client (e.g. STT) SLD SensorML OpenGL SOS Stylers

46. SWE Portrayal Service can “render” to various graphics standards SWE Portrayal Service SLD SensorML KML Collada SOS Google Earth Client Stylers For example, a SWE portrayal service can utilize a SensorML front-end and a Styler back-end to generate graphics content (e.g. KML or Collada)

47. SensorML to Google Earth (KML – Collada) AMSR-E SSM/I MAS TMI LIS

48. SensorML Support Activities

49. SensorML Navigation

50. Current Tool Development and Support for SensorML • SensorML Forum – mail list for SensorML discussion (300+ active members from various backgrounds) https://lists.opengeospatial.org/mailman/listinfo/sensorml • Open Source SensorML Process Execution Engine – Along with open-source process model library, provides execution environment for SensorML described algorithms • Open Source SensorML editor and process chaindevelopment client – on-going development of tools to allow human-friendly editors for SensorML descriptions • SensorML-enabled decision support client – Open source Space Time Toolkit is SensorML-enabled and will be available to discover, access, task, and process sensor observations; use as is or as template for COTS development • SensorML white papers and tutorials – being written and released on an array of SensorML topics • Describing a Simple Sensor System • Creating New Process Models;