260 likes | 356 Views
Using Prioritized Network Traffic to Achieve End-to-End Predictability. BBN Technologies OOMWorks LLC Cambridge, MA Metuchen, NJ Craig Rodrigues Yamuna Krishnamurthy
E N D
Using Prioritized Network Traffic to Achieve End-to-End Predictability • BBN Technologies OOMWorks LLC • Cambridge, MA Metuchen, NJ • Craig Rodrigues Yamuna Krishnamurthy • Irfan Pyarali • Pradeep Gore Real-Time and Embedded Distributed Object Computing • June 15-18, 2002 • Arlington, Virginia, USA
Managed Behavior for Computational Resources • In order to preserve end-to-end Quality of Service (QoS) requirements, different technologies must be used to manage the behavior of computational resources • Processor Resources • RT-CORBA • Network Resources • RSVP • DiffServ
Talk Outline • Real-Time CORBA (RTCORBA) Overview • RSVP and Diffserv Overviews • Diffserv enhancements to RTCORBA middleware • Performance results
Real-time CORBA Overview • RT-CORBA adds QoS control to regular CORBA to improve the application predictability • Bounding priority inversions • Managing resources end-to-end • Policies & mechanisms for resource configuration/control in RT-CORBA include: • Processor Resources • Thread pools • Priority models • Portable priorities • Communication Resources • Protocol policies • Explicit binding • Memory Resources • Request buffering • These capabilities address some (but by no means all) important real-time application development challenges
Preserving Priorities End-to-End • RT-CORBA preserves end-to-end priorities by: • Mapping importance of activities to corresponding OS priorities • Propagating priorities across the network as activity spans multiple hosts • However , RT-CORBA specification is less explicit about: • Communication transport and underlying network • Unless this behavior is carefully considered and modeled: • In-transit activities will be affected by network idiosyncrasies • End-to-end predictability in the system difficult to achieve
What is RSVP? • Resource Reservation Protocol, specified in IETF RFC 2205 • QoS properties are requested by an “out-of-band” signalling protocol • Can be used to request a reserved bandwidth between a sender and receiver host • Information about the reservation is stored in each intermediate router: “soft-state”
Problems with RSVP • Protocol is very complex to implement • much complexity due to anticipated use-cases of large scale multicast trees; multicast not widely used • Storing reservation per-flow reservation state is costly • some work being done into looking reservations for aggregate flows • Microsoft implemented RSVP in Win2K, disabled it in WinXP • equipment vendors have less incentive to implement it
Problems with RSVP (continued) • IETF Next Steps in Signaling (NSIS) Working group • realized that RSVP has it shortcomings, but a signaling protocol for QoS is useful • developing new signaling mechanism (“RSVP Lite”?) • specification due end of 2002 • Maybe worth looking at in future, but right nowindustry seems to be moving away RSVP
Solution: Differentiated Services • Use Differentiated Services to prioritize the RT-CORBA network traffic • Differentiated Services (Diffserv) architecture provides different types or levels of service for network traffic • Diffserv Code Points (DSCP) are added to data packet headers to specify the expected type of service • Diffserv enabled routers and network elements use DSCP codepoints to differentiate the network traffic • We are enhancing the existing TAO ORB's real-time implementation by making it Diffserv aware
What is Diffserv? IP Datagram Header Version (4-bits) Header Length (4-bits) Diffserv Field (8-bits) Total Length (16-bits) Identification (16-bits) Flags (3-bits) Fragment offset (13-bits) TTL (8-bits) Protocol (8-bits) Checksum (16-bits) Source Address (32-bits) Destination Address (32-bits) • Diffserv field: 6 bits of Diffserv Codepoint, 2 bits ECN • Each DSCP (0-63) specifies a Per-Hop-Behavior (PHB), which is a type of router-level QoS (RFC 2475)
Diffserv QoS Enhancements to the TAO ORB • Provided an efficient and flexible way of setting the DSCP codepoints in the ORB data packets • Done by extending the RTCORBA protocol properties • Setting the DSCP codepoint in the message sent and the reply received • Provided a mechanism to map the RT-CORBA priorities to the Diff-Serv network priorities
RTCORBA IDL Modification local interface TCPProtocolProperties : ProtocolProperties { attribute long send_buffer_size; attribute long recv_buffer_size; attribute boolean keep_alive; attribute boolean dont_route; attribute boolean no_delay; attribute boolean enable_network_priority; };
Initialize Protocol Properties //Set the tcp protocol properties RTCORBA::TCPProtocolProperties_var tcp_properties = rt_orb->create_tcp_protocol_properties (send_buffer_size, recv_buffer_size, 1, // keep_alive 0, // dont_route 1, // no_delay enable_network_priority);
Client Side – Initializing the Policy List RTCORBA::ProtocolList protocols; protocols.length (1); protocols[0].protocol_type = 0; protocols[0].transport_protocol_properties = RTCORBA::ProtocolProperties::_duplicate (tcp_properties.in ()); protocols[0].orb_protocol_properties = RTCORBA::ProtocolProperties::_nil (); poa_policy_list[1] = rt_orb->create_client_protocol_policy (protocols );
Client Side - Change Transport Protocol Policy • ORB Level object = orb->resolve_initial_references ("ORBPolicyManager"); CORBA::PolicyManager_var policy_manager = CORBA::PolicyManager::_narrow (object.in () ); policy_manager->set_policy_overrides (policy_list, CORBA::SET_OVERRIDE );
Client Side - Change Transport Protocol Policy • Thread Level orb->orb_core () >policy_current().set_policy_overrides (policy_list, CORBA::SET_OVERRIDE); • Object Level CORBA::Object_var object = server->_set_policy_overrides (policy_list, CORBA::SET_OVERRIDE); server = Test::_narrow (object.in () );
Server Side - Initializing the Policy List RTCORBA::ProtocolList protocols; protocols.length (1); protocols[0].protocol_type = 0; protocols[0].transport_protocol_properties = RTCORBA::ProtocolProperties::_duplicate (tcp_properties.in ()); protocols[0].orb_protocol_properties = RTCORBA::ProtocolProperties::_nil (); poa_policy_list[1] = rt_orb->create_server_protocol_policy (protocols );
Server Side – Set Policy on the POA //Create POA with Diffserv enabled PortableServer::POA_var poa_with_ds = root_poa->create_POA ("POA_WITH_DS", poa_manager.in (), poa_policy_list);
RTCORBA to Network Priority Mapping • Added a pluggable RTCORBA to Network Priority Mapping framework • Similar to the RTCORBA to Thread Priority Mapping framework • RTCORBA IDL modification: typedef short NetworkPriority; native NetworkPriorityMapping; • The framework allows the user to implement their own mapping algorithms and plug it in using the ACE Service Configurator
RTCORBA/Diffserv Testbed 192.168.1.* 192.168.10.* Linux Linux Linux Diffserv Diffserv T1 IIOP IIOP T2 Traffic Congestion Linux Priority of T1 = Priority of T2
Performance Results • No network traffic congestion • With network traffic congestion • With network traffic congestion and DSCP set on Thread 1
Performance Results With congestion traffic and DSCP set on T1
JTIDS Network ISR Constellation UAV PPLI Air Operations Center Air Defense System Integrator With JCPI STU III Back-Up TADIL & Intel Feeds Nellis – ISAFAF Fiber Optical Link Primary UAV Precise Position Location Information Prioritized Data Feeds PPLI GCS #1 GCS #2 GCS #3 GCS #4 Potential Use Case for RT-CORBA/Diffserv “Predator” Squadron Operations Center ADSI with JCPI & PowerScene Equipped Source: UAV Battlelab, Eglin Air Force Base U.S. Air Force
Summary • End-to-end QoS requires resource management of CPU and network resources • RT-CORBA today primarily manages CPU resources • Our enhancement allows RT-CORBA middleware to manage network resourcesusing Diffserv, which is priority based
Obtaining Software • http://deuce.doc.wustl.edu/Download.html • Contacts: • Craig Rodrigues, crodrigu@bbn.com, 1-617-873-4725 • Yamuna Krishnamurthy, yamuna@oomworks.com, 1-732-205-0346
Further References • “Protocol Selection and Explicit Binding”,Schmidt & Vinoskihttp://www.cuj.com/experts/2005/vinoski.htm • “An Architectural for Differentiated Services”, RFC 2475http://www.ietf.org/rfc/rfc2475.txt