Concurrent C++ Network Programming with Patterns & Frameworks

1. Dr. Douglas C. Schmidt d.schmidt@vanderbilt.edu www.dre.vanderbilt.edu/~schmidt/ www.cs.wustl.edu/~schmidt/tutorials-ace.html Concurrent C++ Network Programming with Patterns & Frameworks Professor of EECS Vanderbilt University Nashville, Tennessee

2. Motivation: Challenges of Networked Applications Complexities in networked applications • Accidental Complexities • Low-level APIs • Poor debugging tools • Algorithmic decomposition • Continuous re-invention/discovery of core concepts & components • Inherent Complexities • Latency • Reliability • Load balancing • Causal ordering • Scheduling & synchronization • Deadlock • Observation • Building robust, efficient, & extensible concurrent & networked applications is hard • e.g., we must address many complex topics that are less problematic for non-concurrent, stand-alone applications

3. Presentation Outline Cover OO techniques & language features that enhance software quality • Patterns,which embody reusable software architectures & designs • C++ wrapper facades & frameworks, which simplify concurrent network programming • C++ language features, e.g., classes, dynamic binding & inheritance, parameterized types • Presentation Organization • Background • Concurrent & network challenges & solution approaches • Patterns, wrapper facades, & frameworks in ACE + applications

4. Overview of ACE • Features • Open-source • 200,000+ lines of C++ • 40+ person-years of effort • Ported to many OS platforms • Freely available as open-source • www.cs.wustl.edu/~schmidt/ACE.html • Large user community • www.cs.wustl.edu/~schmidt/ACE-users.html

5. Overview of Patterns • Present solutions to common software problems arising within a certain context • Help resolve key software design forces • Flexibility • Extensibility • Dependability • Predictability • Scalability • Efficiency Client Proxy Service AbstractService service service service • Capture recurring structures & dynamics among software participants to facilitate reuse of successful designs • Generally codify expert knowledge of design strategies, constraints & “best practices” 1 1 The Proxy Pattern

6. Overview of Pattern Languages • Motivation • Individual patterns & pattern catalogs are insufficient • Software modeling methods & tools that just illustrate how, not why, systems are designed • Benefits of Pattern Languages • Define a vocabulary for talking about software development problems • Provide a process for the orderly resolution of these problems • Help to generate & reuse software architectures

7. The Pattern Language for ACE • Pattern Benefits • Preserve crucial design information used by applications & middleware frameworks & components • Facilitate reuse of proven software designs & architectures • Guide design choices for application developers

8. Overview of Frameworks • Frameworks exhibit “inversion of control” at runtime via callbacks • Frameworks provide integrated domain-specific structures & functionality • Frameworks are “semi-complete” applications Application-specific functionality Scientific Visualization Mission Computing E-commerce GUI Networking Database Framework Characteristics www.cs.wustl.edu/~schmidt/frameworks.html

9. Benefits of Frameworks Proxy Proxy Broker Broker OS-AccessLayer ComponentRepository CommunicationServices ComponentConfigurator Broker Broker AdminClient PickingClient PickingControllres AdminControllres PickingViews AdminViews Scheduler/ ActivationList Reactor ServiceRequest ThreadPool LoggingHandler * ServiceRequest ServiceRequest WarehouseRepHalfX • Design reuse • e.g., by guiding application developers through the steps necessary to ensure successful creation & deployment of software Thin UI Clients Distribution Infrastructure Concurrency Infrastructure

10. Benefits of Frameworks • Design reuse • e.g., by guiding application developers through the steps necessary to ensure successful creation & deployment of software • Implementation reuse • e.g., by amortizing software lifecycle costs & leveraging previous development & optimization efforts

11. Benefits of Frameworks • Design reuse • e.g., by guiding application developers through the steps necessary to ensure successful creation & deployment of software • Implementation reuse • e.g., by amortizing software lifecycle costs & leveraging previous development & optimization efforts • Validation reuse • e.g., by amortizing the efforts of validating application- & platform-independent portions of software, thereby enhancing software reliability & scalability www.dre.vanderbilt.edu/scoreboard

12. Comparing Reuse Techniques Naming Events Logging Locking Reactor ADTs Framework Architecture Strings • A framework is an integrated set of classes that collaborate to produce a reusable architecture for a family of applications • Frameworks implement pattern languages INVOKES CALLBACKS Files Locks NETWORKING APPLICATION-SPECIFIC FUNCTIONALITY Component Architecture GUI Middleware Bus • A component is an encapsulation unit with one or more interfaces that provide clients with access to its services • Components can be deployed & configured via assemblies DATABASE Class Library Architecture LOCAL INVOCATIONS APPLICATION- SPECIFIC FUNCTIONALITY Math IPC ADTs • A class is a unit of abstraction & implementation in an OO programming language, i.e., a reusable type that often implements patterns • Classes in class libraries are typically passive Files Strings GUI GLUE CODE EVENT LOOP Locks

13. The Frameworks in ACE ACE frameworks are a product-line architecture for domain of network applications Application- specific functionality Acceptor Connector Stream Component Configurator Reactor Task Proactor

14. Networked Logging Service Example • Key Participants • Client application processes • Generate log records • Client logging daemons • Buffer log records & transmit them to the server logging daemon • Server logging daemon • Receive, process, & store log records • C++ code for all logging service examples are in • ACE_ROOT/examples/ C++NPv1/ • ACE_ROOT/examples/ C++NPv2/

15. Patterns in the Networked Logging Service Leader/ Followers Monitor Object Active Object Half-Sync/ Half-Async Reactor Pipes & Filters Acceptor- Connector Component Configurator Proactor Wrapper Facade Thread-safe Interface Strategized Locking Scoped Locking

16. ACE Socket Wrapper Façade Classes • ACE defines a set of C++ classes that address the limitations with the Socket API • Enhance type-safety • Ensure portability • Simplify common use cases • Building blocks for higher-level abstractions These classes are designed in accordance with the Wrapper Facade design pattern

17. The Wrapper Façade Pattern (1/2) Applications Solaris VxWorks Win2K Linux LynxOS • Context • Networked applications must manage a variety of OS services, including processes, threads, socket connections, virtual memory, & files • OS platforms provide low-level APIs written in C to access these services • Problem • The diversity of hardware & operating systems makes it hard to build portable & robust networked application software • Programming directly to low-level OS APIs is tedious, error-prone, & non-portable

18. The Wrapper Façade Pattern (2/2) calls API FunctionA() calls methods Application calls API FunctionB() calls API FunctionC() Wrapper Facade void method1(){ void methodN(){ functionA(); functionA(); data functionB(); } } method1() … methodN() : Application : Wrapper : APIFunctionA : APIFunctionB Facade method() functionA() functionB() • Solution • Apply the Wrapper Facade design pattern (P2) to avoid accessing low-level operating system APIs directly This pattern encapsulates data & functions provided by existing non-OO APIs within more concise, robust, portable, maintainable, & cohesive OO class interfaces

19. Roles in the ACE Socket Wrapper Facade • The active connection role (ACE_SOCK_Connector) is played by a peer application that initiates a connection to a remote peer • The passive connection role (ACE_SOCK_Acceptor) is played by a peer application that accepts a connection from a remote peer & • The communication role (ACE_SOCK_Stream) is played by both peer applications to exchange data after they are connected

20. The ACE_SOCK_Connector Class • Motivation • There is a confusing asymmetry in the Socket API between (1) connection roles & (2) socket modes • e.g., an application may accidentally call recv() or send() on a data-mode socket handle before it's connected • This problem can't be detected until run time since C socket handles are weakly-typed int buggy_echo_client (u_short port_num, const char *s) { int handle = socket (PF_UNIX, SOCK_DGRAM, 0); write (handle, s, strlen (s) + 1); sockaddr_in s_addr; memset (&s_addr, 0, sizeof s_addr); s_addr.sin_family = AF_INET; s_addr.sin_port = htons (port_num); connect (handle, (sockaddr *) &s_addr, sizeof s_addr); } Operations called in wrong order

21. The ACE_SOCK_Connector Class • Class Capabilities • ACE_SOCK_Connector is factory that establishes a new endpoint of communication actively & provides capabilities to • Initiate a connection with a peer acceptor & then to initialize an ACE_SOCK_Stream object after the connection is established • Initiate connections in either a blocking, nonblocking, or timed manner • Use C++ traits to support generic programming techniques that enable wholesale replacement of IPC functionality

22. Using the ACE_SOCK_Connector (1/3) • This example shows how the ACE_SOCK_Connector can be used to connect a client application to a Web server int main (int argc, char *argv[]) { const char *pathname = argc > 1 ? argv[1] : "index.html"; const char *server_hostname = argc > 2 ? argv[2] : “www.dre.vanderbilt.edu"; typedef ACE_SOCK_Connector CONNECTOR; CONNECTOR connector; typename CONNECTOR::PEER_STREAM peer; typename CONNECTOR::PEER_ADDR peer_addr; if (peer_addr.set (80, server_hostname) == -1) return 1; else if (connector.connect (peer, peer_addr) == -1) return 1; • Instantiate the connector, data transfer, & address objects • Block until connection established or connection request failure

23. Using the ACE_SOCK_Connector (2/3) // Designate a nonblocking connect. if (connector.connect (peer, peer_addr, &ACE_Time_Value::zero) == -1) { if (errno == EWOULDBLOCK) { // Do some other work ... // Now, try to complete connection establishment, // but don't block if it isn't complete yet. if (connector.complete (peer, 0, &ACE_Time_Value::zero) == -1) • Perform a non-blocking connect • If connection not established, do other work & try again without blocking // Designate a timed connect. ACE_Time_Value timeout (10); // 10 second timeout. if (connector.connect (peer, peer_addr, &timeout) == -1) { if (errno == ETIME) { // Timeout, do something else • Perform a timed connect e.g., 10 seconds in this case

24. The ACE_SOCK_Stream Class (1/2) • Motivation • Developers can misuse sockets in ways that can't be detected during compilation • An ACE_SOCK_Stream object can't be used in any role other than data transfer without violating its (statically type-checked) interface int buggy_echo_server (u_short port_num) { sockaddr_in s_addr; int acceptor = socket (PF_UNIX, SOCK_DGRAM, 0); s_addr.sin_family = AF_INET; s_addr.sin_port = port_num; s_addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, sizeof s_addr); int handle = accept (acceptor, 0, 0); for (;;) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; write (handle, buf, n); } } Reading from wrong handle

25. The ACE_SOCK_Stream Class (2/2) • Class Capabilities • Encapsulates data transfer mechanisms supported by data-mode sockets to provide the following capabilities: • Support for sending & receiving up to n bytes or exactly n bytes • Support for “scatter-read,” which populate multiple caller-supplied buffers instead of a single contiguous buffer • Support for ``gather-write'' operations, which transmit the contents of multiple noncontiguous data buffers in a single operation • Support for blocking, nonblocking, & timed I/O operations • Support for generic programming techniques that enable the wholesale replacement of functionality via C++ parameterized types

26. Using the ACE_SOCK_Stream (1/2) • This example shows how an ACE_SOCK_Stream can be used to send & receive data to & from a Web server // ...Connection code from example in Section 3.5 omitted... char buf[BUFSIZ]; iovec iov[3]; iov[0].iov_base = (char *) "GET "; iov[0].iov_len = 4; // Length of "GET ". iov[1].iov_base = (char *) pathname; iov[1].iov_len = strlen (pathname); iov[2].iov_base = (char *) " HTTP/1.0\r\n\r\n"; iov[2].iov_len = 13; // Length of " HTTP/1.0\r\n\r\n"; if (peer.sendv_n (iov, 3) == -1) return 1; for (ssize_t n; (n = peer.recv (buf, sizeof buf)) > 0; ) ACE::write_n (ACE_STDOUT, buf, n); return peer.close () == -1 ? 1 : 0; } • Initialize the iovec vector for scatter-read & gather-write I/O • Perform blocking gather-write on ACE_SOCK_Stream • Perform blocking read on ACE_SOCK_Stream

27. Using the ACE_SOCK_Stream (2/2) • Blocking & non-blocking I/O semantics can be controlled via the ACE_SOCK_STREAM enable() & disable() methods, e.g., • peer.enable (ACE_NONBLOCK); // enables non blocking peer.disable (ACE_NONBLOCK); // disable non blocking • If the I/O operation blocks, it returns a -1 & errno is set to EWOULDBLOCK • I/O operations can involve timeouts, e.g., ACE_Time_Value timeout (10); // 10 second timeout If (peer.sendv_n (iov, 3, &timeout) == -1) { // check if errno is set to ETIME, // which indicates a timeout } // similarly use timeout for receiving data

28. The ACE_SOCK_Acceptor Class (1/2) • Motivation • The C functions in the Socket API are weakly typed, which makes it easy to apply them incorrectly in ways that can’t be detected until run-time • The ACE_SOCK_Acceptor class ensures type errors are detected at compile-time int buggy_echo_server (u_short port_num) { sockaddr_in s_addr; int acceptor = socket (PF_UNIX, SOCK_DGRAM, 0); s_addr.sin_family = AF_INET; s_addr.sin_port = port_num; s_addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, sizeof s_addr); int handle = accept (acceptor, 0, 0); for (;;) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; write (handle, buf, n); } } Reading from wrong handle

29. The ACE_SOCK_Acceptor Class (2/2) • Class Capabilities • This class is a factory that establishes a new endpoint of communication passively & provides the following capabilities: • It accepts a connection from a peer connector & then initializes an ACE_SOCK_Stream object after the connection is established • Connections can be accepted in either a blocking, nonblocking, or timed manner • C++ traits are used to support generic programming techniques that enable the wholesale replacement of functionality via C++ parameterized types

30. Using the ACE_SOCK_Acceptor • This example shows how an ACE_SOCK_Acceptor & ACE_SOCK_Stream can be used to accept connections & send/receive data to/from a web client extern char *get_url_pathname (ACE_SOCK_Stream *); int main () { ACE_INET_Addr server_addr; ACE_SOCK_Acceptor acceptor; ACE_SOCK_Stream peer; if (server_addr.set (80) == -1) return 1; if (acceptor.open (server_addr) == -1) return 1; for (;;) { if (acceptor.accept (peer) == -1) return 1; peer.disable (ACE_NONBLOCK); // Ensure blocking <send_n>. ACE_Auto_Array_Ptr<char *> pathname (get_url_pathname (peer)); ACE_Mem_Map mapped_file (pathname.get ()); if (peer.send_n (mapped_file.addr (), mapped_file.size ()) == -1) return 1; peer.close (); } return acceptor.close () == -1 ? 1 : 0; } • Instantiate the acceptor, data transfer, & address objects • Initialize a passive mode endpoint to listen for connections on port 80 • Accept a new connection • Send the requested data • Close the connection to the sender • Stop receiving any connections

31. The ACE_Message_Block Class (1/2) MESSAGES BUFFERED AWAITING PROCESSING MESSAGES BUFFERED FOR TRANSMISSION MESSAGES IN TRANSIT • Motivation • Many networked applications require a means to manipulate messages efficiently, e.g.: • Storing messages in buffers as they are received from the network or from other processes • Adding/removing headers/trailers from messages as they pass through a user-level protocol stack • Fragmenting/reassembling messages to fit into network MTUs • Storing messages in buffers for transmission or retransmission • Reordering messages that were received out-of-sequence

32. The ACE_Message_Block Class (2/2) • Class Capabilities • This class is a composite that enables efficient manipulation of messages via the following operations: • Each ACE_Message_Block contains a pointer to a reference-counted ACE_Data_Block which in turn points to the actual data associated with a message • It allows multiple messages to be chained together into a composite message • It allows multiple messages to be joined together to form an ACE_Message_Queue • Ittreats synchronization & memory management properties as aspects

33. Two Kinds of Message Blocks • Simple messages contain a one ACE_Message_Block • An ACE_Message_Block points to an ACE_Data_Block • An ACE_Data_Block points to the actual data payload • Composite messages contain multiple ACE_Message_Blocks • These blocks are linked together in accordance with the Composite pattern • Composite messages often consist of a control message that contains bookkeeping information • e.g., destination addresses, followed by one or more data messages that contain the actual contents of the message • ACE_Data_Blocks can be referenced counted

34. ACE CDR Streams • Motivation • Networked applications that send/receive messages often require support for • Linearization • To handle the conversion of richly typed data to/from raw memory buffers • (De)marshaling • To interoperate with heterogeneous compiler alignments & hardware instructions with different byte-orders • The ACE_OutputCDR & ACE_InputCDR classes provide a highly optimized, portable, & convenient means to marshal & demarshal data using the standard CORBA Common Data Representation (CDR) • ACE_OutputCDR creates a CDR buffer from a data structure (marshaling) • ACE_InputCDR extracts data from a CDR buffer (demarshaling)

35. The ACE_OutputCDR & ACE_InputCDR Classes • Class Capabilities • ACE_OutputCDR & ACE_InputCDR support the following features: • They provide operations to (de)marshal the following types: • Primitive types, e.g., booleans; 16-, 32-, & 64-bit integers; 8-bit octets; single & double precision floating point numbers; characters; & strings • Arrays of primitive types • The insertion (<<) and extraction (>>) operators can marshal & demarshal primitive types, using the same syntax as the C++ iostream components • ACE_Message_Block chains are used internally to minimize mem copies • They take advantage of CORBA CDR alignment & byte-ordering rules to avoid memory copying & byte-swapping operations, respectively • They provide optimized byte swapping code that uses inline assembly language instructions for common hardware platforms (such as Intel x86) & standard hton*()& ntoh*() macros/functions on other platforms • They support zero copy marshaling & demarshaling of octet buffers • Users can define custom character set translators for platforms that do not use ASCII or Unicode as their native character sets

36. Log Record Message Structure • This example uses a 8-byte, CDR encoded header followed by the payload • Header includes byte order, payload length, & other fields ACE_Log_Record is a type that ACE uses internally to keep track of the fields in a log record class ACE_Log_Record { private: ACE_UINT type_; ACE_UINT pid_; ACE_Time_Value timestamp_; char msg_data_[ACE_MAXLOGMSGLEN]; public: ACE_UINT type () const; ACE_UINT pid () const; const ACE_Time_Value timestamp () const; const char *msg_data () const; };

37. Using ACE_OutputCDR • We show the ACE CDR insertion (operator<<) & extraction (operator>>) operators for ACE_Log_Record that's used by client application & logging server int operator<< (ACE_OutputCDR &cdr, const ACE_Log_Record &log_record) { size_t msglen = log_record.msg_data_len (); // Insert each <log_record> field into the output CDR stream. cdr << ACE_CDR::Long (log_record.type ()); cdr << ACE_CDR::Long (log_record.pid ()); cdr << ACE_CDR::Long (log_record.time_stamp ().sec ()); cdr << ACE_CDR::Long (log_record.time_stamp ().usec ()); cdr << ACE_CDR::ULong (msglen); cdr.write_char_array (log_record.msg_data (), msglen); return cdr.good_bit (); } After marshaling all the fields of the log record into the CDR stream, return the success/failure status

38. Using ACE_InputCDR int operator>> (ACE_InputCDR &cdr, ACE_Log_Record &log_record) { ACE_CDR::Long type; ACE_CDR::Long pid; ACE_CDR::Long sec, usec; ACE_CDR::ULong buffer_len; // Extract each field from input CDR stream into <log_record>. if ((cdr >> type) && (cdr >> pid) && (cdr >> sec) && (cdr >> usec) && (cdr >> buffer_len)) { ACE_TCHAR log_msg[ACE_Log_Record::MAXLOGMSGLEN + 1]; log_record.type (type); log_record.pid (pid); log_record.time_stamp (ACE_Time_Value (sec, usec)); cdr.read_char_array (log_msg, buffer_len); log_msg[buffer_len] = '\0'; log_record.msg_data (log_msg); } return cdr.good_bit (); } Temporaries used during demarshaling (not always necessary) After demarshaling all the fields of the log record from the CDR stream, return the success/failure status

39. The ACE Reactor Framework • Motivation • Many networked applications are developed as event-driven programs • Common sources of events in these applications include activity on an IPC stream for I/O operations, POSIX signals, Windows handle signaling, & timer expirations • To improve extensibility & flexibility, it’s important to decouple the detection, demultiplexing, & dispatching of events from the handling of events

40. The ACE Reactor Framework • The ACE Reactor framework implements the Reactor pattern (POSA2) • This pattern & framework automates the • Detection of events from various sources of events • Demultiplexing the events to pre-registered handlers of these events • Dispatching to hook methods defined by the handlers to process the events in an application-defined manner

41. The ACE Reactor Framework • The classes in the ACE Reactor framework implement the Reactor pattern:

42. The Reactor Pattern Participants Reactor Event Handler * handle_events() register_handler() remove_handler() dispatches handle_event () get_handle() * owns Handle * notifies handle set <<uses>> Concrete Event Handler A Concrete Event Handler B Synchronous Event Demuxer handle_event () get_handle() handle_event () get_handle() select () • The Reactor architectural pattern allows event-driven applications to demultiplex & dispatch service requests that are delivered to an application from one or more clients

43. The Reactor Pattern Dynamics : Main Program : Concrete : Reactor : Synchronous Event Handler Event Demultiplexer Con. Event Events register_handler() Handler get_handle() Handle handle_events() Handles select() event handle_event() Handles service() • Observations • Note inversion of control • Also note how long-running event handlers can degrade the QoS since callbacks steal the reactor’s thread!

44. The ACE_Event_Handler Class (1/2) • Motivation • Networked applications are often “event driven” • i.e., their processing is driven by callbacks • There are problems with implementing callbacks by defining a separate function for each type of event function1 data1 function2 data2 function3 data3 Demultiplexer Event Sources • It is therefore more effective to devise an “object-oriented” event demultiplexing mechanism • This mechanism should implement callbacks via object-oriented event handlers

45. The ACE_Event_Handler Class (2/2) • Class Capabilities • This base class of all reactive event handlers provides the following capabilities: • It defines hook methods for input, output, exception, timer, & signal events • Its hook methods allow applications to extend event handler subclasses in many ways without changing the framework • Its use of object-oriented callbacks simplifies the association of data with hook methods that manipulate the data • Its use of objects also automates the binding of an event source (or set of sources) with data the event source is associated with, such as a network session • It centralizes how event handlers can be destroyed when they're not needed • It holds a pointer to the ACE_Reactor that manages it, making it simple for an event handler to manage its event (de)registration correctly

46. The ACE_Event_Handler Class API This class handles variability of event processing behavior via a common event handler API

47. Types of Events & Event Handler Hooks • When an application registers an event handler with a reactor, it must indicate what type(s) of event(s) the event handler should process • ACE designates these event types via enumerators defined in ACE_Event_Handlerthat are associated with handle_*()hook methods • These values can be combined (``or'd'' together) to efficiently designate a set of events • This set of events can populate the ACE_Reactor_Mask parameter that's passed to the ACE_Reactor::register_handler() methods

48. Event Handler Hook Method Return Values • When registered events occur, the reactor dispatches the appropriate event handler's handle_*() hook methods to process them • When a handle_*() method finishes its processing, it must return a value that's interpreted by the reactor as follows: • Before the reactor removes an event handler, it invokes the handler's hook method handle_close(), passing ACE_Reactor_Mask of the event that's now unregistered

49. Using the ACE_Event_Handler Class (1/8) • We implement our logging server by inheriting from ACE_Event_Handler & driving its processing via the reactor’s event loop to handle two types of events: • Data events, which indicate the arrival of log records from connected client logging daemons • Accept events, which indicate the arrival of new connection requests from client logging daemons Logging Event Handler Logging Event Handler Logging Acceptor ACE_Reactor

50. Using the ACE_Event_Handler Class (2/8) • We define two types of event handlers in our logging server: • Logging_Event_Handler • Processes log records received from a connected client logging daemon • Uses the ACE_SOCK_Stream to read log records from a connection • Logging_Acceptor • A factory that allocates a Logging_Event_Handler dynamically & initializes it when a client logging daemon connects • Uses ACE_SOCK_Acceptor to initialize ACE_SOCK_Stream contained in Logging_Event_Handler