Deploying Web Services: Bringing NNEC To The Edge

Deploying Web Services: Bringing NNEC To The Edge Presentation to MCC 2006 James Busch Principal Scientist NC3A The Hague NATO UNCLASSIFIED

Deploying Web Services: Bringing NNEC to the Edge Brief Intro to NNEC Brief Intro to SOA and Web Services Deployed Web Services Experiment Conclusions NATO UNCLASSIFIED

NNEC? • “It is a global, Web-enabled platform for multiple forms of collaboration. This platform enables individuals, groups, companies and universities anywhere in the world to collaborate - for the purposes of innovation, production, education, research, entertainment [and] war-making - like no creative platform before.” New York Times columnist Thomas L. Friedman on IT’s Effect on Globalization, 2006 NATO UNCLASSIFIED

NNEC! • “Future operations will be more complex and multidimensional. The planning and execution of operations will ... [require] truly interoperable forces. To support this end, the use of Alliance forces must change from a pattern of de-confliction to one of integration where emerging technologies and concepts, like network-enabled capability, are increasingly used.” NATO’s Strategic Commanders, 2004 NATO UNCLASSIFIED

NNEC Challenge • Information comes from many sources and is needed by many different users. It must be: • Securely Shared • Commonly Understood • Integrated and Analyzed ? …but how? NATO UNCLASSIFIED

NNEC Feasibility StudyAmbitions • Identify types of C2ISR capabilities required to ‘enable’ NATO Network Centric Operations • Based on agreed Missions, Concepts of Operations, involving use of Modular Force Structures such as the NRF (NATO Response Force) • Utilization of Capability Based Planning approach to identify ‘mix’ of National and NATO capabilities • Develop a Strategy and Roadmap for realization of a Networking and Information Infrastructure (NII) • To ‘enable’ required C2ISR capabilities and to support the broader needs of the Alliance. • Involves networking of national funded, NATO common funded and multi-national funded capabilities NATO UNCLASSIFIED

Main Technical Components of NII • Communications • Information and Integration Services • Information Assurance • Service Management NATO UNCLASSIFIED

Information SystemsRecommendations for the NII • Service Oriented Architecture • Loose coupling, dynamic discovery, service oriented • Open and accepted standards • XML, SOAP, UDDI, WS-Security, etc. • “Federation of Systems” vice “System of Systems” Concept • Autonomous Systems • Federated Services • National Centric – no NATO “GIG” • Flexible level of national participation (CMM concept) • Incorporation of legacy systems • Interface to – as opposed to eliminate – existing capability • This includes new “Value Added” services such as REP • Metadata supported information sharing and exchange • XML as common language • Development of “ontologies” to support information sharing • Usage of NATO XML Registry NATO UNCLASSIFIED

Service Oriented Architecture • Service Oriented Architecture is the future of system development: “…the single most important theme in modern application development is service oriented architecture…” – Gartner Group • Recommended in NNEC Feasibility Study! • Information is provided as a series of “Services” – which can be as granular as you like • A “Consumer” requests information from a “Producer” (Service) and receives a response with some kind of data • Benefits • Rapidly integrate new and existing systems into the ‘sharing network’  maximize interoperability. • Utilize ‘open source’ and COTS that support ‘open standards’  minimize cost and reduce/eliminate vendor dependencies. • Securely share information across various security levels amongst various coalition partners. • Reduced integration costs and time compared with current approaches and technologies. • Web Services – an instantiation of SOA – holds promise for solving traditional integration problems using commonly available technologies – XML, HTTP, TCP/IP, etc. – and enabling integration with minimal or no changes to underlying (new or existing) systems NATO UNCLASSIFIED

Service Producer Service Consumer Data and applications available for use accessible via services. Metadata added to services based on producer’s format. Automated search of data services using metadata. Pulls data of interest. Based on producer registered format and definitions, translates into needed structure. • Describes content using metadata • Posts metadata in catalogs for discovery • Exposes data and applications as services Messaging Services Data Services Service Enabled Infrastructure Security Services Monitoring Services Registry Services Transformation Services What is Service Oriented Architecture? Request/Invoke • Searches metadata catalogs to find data services • Analyzes metadata search results found • Pulls selected data based on metadata understanding Publish Discover NATO UNCLASSIFIED

SOA  Web Services • SOA is an enterprise architecture • Web Services is an implementation of that architecture… • …following a common suite of standards (W3C, OASIS, etc.) • XML for data • SOAP for messaging • HTTP over TCP for transport • UDDI for registering services • WSDL for describing services • Can be used to open a controlled “window” into an existing application (aka the Web Service interface) NATO UNCLASSIFIED

Web Services – How they work Service (Information Provider) Web Services Interface Services come “on line” (join the network) Network Service (Information Provider) Web Services Interface Web Services Interface Service (Information Provider) Registry (Discovery Service) Security Service NATO UNCLASSIFIED

Web Services Pattern 1:Synchronous Request-Response • Consumer makes request from Service and waits for response • No state (persistence) – Service “forgets” about Consumer as soon as response is sent • Pluses • Simple, most basic model • Easy to implement (any web browser can be a client) • Loose coupling between Client and Service • Minuses • Client waits on Service • Data is only received when asked for (not good for time-dependent data) NATO UNCLASSIFIED

Web Services Pattern 1:Synchronous Request-Response • Useful for: • Services where the information is not time-dependent (e.g. document repositories) • Situations where Clients only want data when it asks • When simplicity is important • Situations where Client and Service(s) have no knowledge of each other NATO UNCLASSIFIED

Web Services Pattern 2: Asynchronous “Push” (aka Publish-Subscribe) • Consumer asks Service to send data whenever it is available (“subscribes” to Service) • Consumer implements a “listener”; Service implements capability to initiate messages (not just respond) • Pluses • Data is “pushed” out as soon as it is available, and only when it changes • Client doesn’t have to wait on service • Less back-and-forth between client and server? (more on this later…) • Minuses • Both Consumer and Service must implement capability beyond the basic • Consumer and Service are forced to “know” about each other NATO UNCLASSIFIED

Web Services Pattern 2:Asynchronous “Push” • Useful for: • Service offering very time-sensitive information (e.g. (near)real-time tracks) • Client who wants new information as soon as it is available without having to ask for it • Reducing the number of queries between Client and Service? NATO UNCLASSIFIED

The Overhead of Web Services • “XML/SOAP Adds So Much Overhead!” <?xml version=…> <SOAP-ENV:Envelope …> <SOAP-ENV:Body…> <XML Content…> Payload goes here… </XML Content> </SOAP-ENV:Body> </SOAP-ENV:Envelope> • The Disadvantage Is: • Lots of overhead from all the tags, headers, white space • The Benefits Are: • Structured information via XML (vital for information sharing, interoperability) • SOAP Envelope used for WS-Security, WS-Federation, etc. NATO UNCLASSIFIED

The Overhead of Web Services (2) • XML “Efficiency” • Huge advances in last few years • W3C Efficient XML Interchange Working Group working on standard for efficient XML transmission • Reduce XML documents by factor of 100 • Maintain all the benefits of XML, mostly lose the drawback • But… • Digital Signatures and Encryption remain problematic • Add more overhead for security • Don’t compress well • XML is still not appropriate for every situation • Very small messages • Streaming data NATO UNCLASSIFIED

The Network Effect of Web Services Two factors affect the performance of Web Services • Throughput: The number of requests handled in a given time period by the service (measured on server side) • Latency: The round-trip time between sending the request and receiving a response (throughput + network transit). Impacted by: • Amount of bandwidth available (real and effective) • Message size/composition • Additional overhead imposed by HTTP NATO UNCLASSIFIED

Web Services at the “Edge”? • Key component in transformed NATO is “lightweight” (tactical) client • Mobile user • Deployed CJTF • So what are effects on Web Service performance of • Reduced network bandwidth • Network congestion • Larger/smaller volumes of data NATO UNCLASSIFIED

EPW1038 at CWID 2006 • Experiment designed to test variety of Web Service calls in a variety of network conditions • Query returning small amounts of data (small request, small response) • Query returning large amounts of data (small request, large response) • Publish large amounts of data (large request, small response) • Subscribe to “pushed” track and weather information (potentially very large amounts of data) • Related to RTO-061 experiment described yesterday • Where are inefficiencies and what (if anything) can be done? NATO UNCLASSIFIED

Network 1: Local LAN • Clients and all Services local to CWID (Norway) • Network capacity of 100 Mbps NATO UNCLASSIFIED

Network 2: Remote Services via CFBLNet • Clients local to CWID (Norway) • Services remote at NC3A (Netherlands) accessed via CFBLNet backbone • Network capacity of +/- 2 Mbps NATO UNCLASSIFIED

Network 3: Remote Services via UMTS • Clients local to CWID (Norway) • Services remote at NC3A (Netherlands) accessed via UMTS (mobile phone network) • Network capacity of 30 - 300 Kbps NATO UNCLASSIFIED

Network Instrument Observer NATO UNCLASSIFIED

Web Service Simple Query REQUEST Service Client RESPONSE NATO UNCLASSIFIED

Web Services Communicationson a Simple Query Initiation Service Client Acknowledgement (1st phase handshake) Acknowledgement (2nd phase handshake) REQUEST Acknowledgment RESPONSE Acknowledgment Termination Termination Acknowledgement Fact: Network performance is strongly influenced by the number of packets! Conclusion: Web Services back-and-forth adds significant traffic in the form of many small packets. Reducing these small messages helps increase efficiency. NATO UNCLASSIFIED

Network Capacity Effectson Web Service performance Web Services worked well until the worst bandwidth/message size configuration. Performance degradation is not linear but logarithmic; trends for large and small messages are nearly identical. Conclusion: Until the “pipe” becomes very small (relative to message size) the size of the message has less impact than the number of packets. NATO UNCLASSIFIED

Overhead of HTTP/SOAPon a Simple Query Web Services (with SOAP and HTTP headers, plus the various small messages in the conversation) can add significant overhead Conclusion: The impact is much less on larger messages than on small. Configure for larger packet sizes where possible. NATO UNCLASSIFIED

Web Service Publish-Subscribe Initiation Service Client Acknowledgement (1st phase handshake) Acknowledgement (2nd phase handshake) REQUEST (“Subscribe”) Acknowledgment RESPONSE RESPONSE RESPONSE RESPONSE Acknowledgment RESPONSE Acknowledgment Acknowledgment Acknowledgment Acknowledgment NATO UNCLASSIFIED

Overhead of Multicast (“push”) vs. Request-Response When data is “push”ed asynchronously, there are less packets in the conversation – generally an initiation, the message itself, and an acknowledgment from the recipient (client). Conclusion: “Push” of data asynchronously reduces HTTP/SOAP overhead because there are less messages in the “conversation”. NATO UNCLASSIFIED

Conclusions • NNEC FS has suggested an NII architecture featuring • Service Oriented Architecture (implemented as Web Services where feasible) • Loosely coupled, dynamically discoverable systems (services) • Extremely valuable interoperability tool in the “enterprise” environment (strategic, operational) • On-going work already moving in this direction(including operationally) • Tests showed Web Services remained functional at low network capabilities • Except once message volume far exceeded available bandwidth • Appropriate for really low bandwidth (highly tactical) settings? Still uncertain… • Number of packets on network had bigger impact than larger request or response messages • Minimize the HTTP “conversations” where possible • Tweak HTTP settings (e.g. MTU) to enable fewer, larger packets • Eliminate superfluous messages (e.g. DNS pings) from network • Multi-cast (i.e. “push”) Web Services have a network performance advantage over request-response • Not appropriate for all situations NATO UNCLASSIFIED

Questions ? NATO UNCLASSIFIED

NC3A Brussels Visiting address: Bâtiment ZAvenue du Bourget 140B-1110 BrusselsTelephone +32 (0)2 7074111Fax +32 (0)2 7078770 Postal address:NATO C3 AgencyBoulevard Leopold IIIB-1110 Brussels - Belgium NC3A The Hague Visiting address: Oude Waalsdorperweg 612597 AK The HagueTelephone +31 (0)70 3743000Fax +31 (0)70 3743239 Postal address:NATO C3 AgencyP.O. Box 1742501 CD The HagueThe Netherlands Contacting NC3A NATO UNCLASSIFIED

Deploying Web Services: Bringing NNEC To The Edge