Mastering Complexity with ACE & Patterns - A Guide for C++ Network Programming

1. Dr. Douglas C. Schmidt d.schmidt@vanderbilt.edu www.cs.wustl.edu/~schmidt/tutorials-ace.html C++ Network ProgrammingMastering Complexity with ACE & Patterns Professor of EECS Vanderbilt University Nashville, Tennessee

2. Introduction (1/3) • ACE • Adaptive Communication Environment • The key contribution: Creation of uniform models which capture a broad spectrum of services • Easier to do on a specific domain • Easier to do when dealing with small building blocks • For instance: The scoped-locking constructs • The motivation: Development of a concurrent server program (socket-based)

3. Comments (2/3) • Huge scope • General overview • Specific inspection of certain parts • Our goal: A brief introduction • (Slides per minute – High rate) • API Specifications (Doxygen) • http://www.dre.vanderbilt.edu/Doxygen/Current/html/ace/hierarchy.html

4. Introduction (3/3) • ACE is somewhat outdated • does not use exceptions • C++ oriented • Some of the ideas seem trivial to Java programmers • Some of the ideas cannot be supported on Java • Thru ACE we can understand the design rationale • Useful for C++ developers • Useful for any developer of concurrent/distributed programs • Example: Java’s thread class is a realization of ACE’s Active Objects pattern • (see next slide)

5. Active Object: intent Decouples method execution from method invocation and simplifies synchronized access to shared resources by concurrent threads

6. Terminology • Framework • Toolkit • Patterns

7. Motivation: Challenges of Networked Applications Complexities in networked applications • Accidental Complexities • Low-level APIs • Poor debugging tools • Algorithmic decomposition • Continuous re-invention/discovery of core concepts & components • Inherent Complexities • Latency • Reliability • Load balancing • Causal ordering • Scheduling & synchronization • Deadlock • Observation • Building robust, efficient, & extensible concurrent & networked applications is hard • e.g., we must address many complex topics that are less problematic for non-concurrent, stand-alone applications

8. Key Capabilities Provided by ACE Event Handling & IPC Service Access & Control Synchronization Concurrency

9. The Layered Architecture of ACE www.cs.wustl.edu/~schmidt/ACE.html • Features • Open-source • 200,000+ lines of C++ • 40+ person-years of effort • Ported to many OS platforms

10. The Pattern Language for ACE • Pattern Benefits • Preserve crucial design information used by applications & middleware frameworks & components • Facilitate reuse of proven software designs & architectures • Guide design choices for application developers

11. The Frameworks in ACE

12. Example: Applying ACE in Real-time Avionics Key Results • Test flown at China Lake NAWS by Boeing OSAT II ‘98, funded by OS-JTF • www.cs.wustl.edu/~schmidt/TAO-boeing.html • Also used on SOFIA project by Raytheon • sofia.arc.nasa.gov • First use of RT CORBA in mission computing • Drove Real-time CORBA standardization • Goals • Apply COTS & open systems to mission-critical real-time avionics • Key System Characteristics • Deterministic & statistical deadlines • ~20 Hz • Low latency & jitter • ~250 usecs • Periodic & aperiodic processing • Complex dependencies • Continuous platform upgrades

13. Limitations with the Socket APIs (1/2) • Poorly structured, non-uniform, & non-portable • API is linear rather than hierarchical • i.e., the API is not structured according to the different phases of connection lifecycle management and the roles played by the participants • No consistency among the names • Non-portable & error-prone • Function names: read() & write() used for any I/O handle on Unix but Windows needs ReadFile() & WriteFile() • Function semantics: different behavior of same function on different OS e.g., accept () can take NULL client address parameter on Unix/Windows, but will crash on some operating systems, such as VxWorks • Socket handle representations: different platforms represent sockets differently e.g., Unix uses unsigned integers whereas Windows uses pointers • Header files: Different platforms use different names for header files for the socket API

14. Limitations with the Socket APIs (2/2) • Lack of type safety • I/O handles are not amenable to strong type checking at compile time • e.g., no type distinction between a socket used for passive listening & a socket used for data transfer • Steep learning curve due to complex semantics • Multiple protocol families & address families • Options for infrequently used features such as broadcasting, async I/O, non blocking I/O, urgent data delivery • Communication optimizations such as scatter-read & gather-write • Different communication and connection roles, such as active & passive connection establishment, & data transfer • Too many low-level details • Forgetting to use the network byte order before data transfer • Possibility of missing a function, such as listen() • Possibility of mismatch between protocol & address families • Forgetting to initialize underlying C structures e.g., sockaddr • Using a wrong socket for a given role

15. Example of Socket API Limitations (1/3) 1 #include <sys/types.h> 2 #include <sys/socket.h> 3 4 const int PORT_NUM = 10000; 5 6 int echo_server () 7 { 8 struct sockaddr_in addr; 9 int addr_len; 10 char buf[BUFSIZ]; 11 int n_handle; 12 // Create the local endpoint. Possible differences in header file names Forgot to initialize to sizeof (sockaddr_in) Use of non-portable handle type

16. Example of Socket API Limitations (2/3) 13 int s_handle = socket (PF_UNIX, SOCK_DGRAM, 0); 14 if (s_handle == -1) return -1; 15 16 // Set up address information where server listens. 17 addr.sin_family = AF_INET; 18 addr.sin_port = PORT_NUM; 19 addr.sin_addr.addr = INADDR_ANY; 20 21 if (bind (s_handle, (struct sockaddr *) &addr, 22 sizeof addr) == -1) 23 return -1; 24 Use of non-portable return value Protocol and address family mismatch Wrong byte order Unused structure members not zeroed out Missed call to listen()

17. Example of Socket API Limitations (3/3) 25 // Create a new communication endpoint. 26 if (n_handle = accept (s_handle, (struct sockaddr *) &addr, 27 &addr_len) != -1) { 28 int n; 29 while ((n = read (s_handle, buf, sizeof buf)) > 0) 30 write (n_handle, buf, n); 31 32 close (n_handle); 33 } 34 return 0; 35 } SOCK_DGRAM handle illegal here Reading from wrong handle No guarantee that “n” bytes will be written

18. The Wrapper Facade Pattern (1/2) Applications Solaris VxWorks Win2K Linux LynxOS • Context • Networked applications must manage a variety of OS services, including processes, threads, socket connections, virtual memory, & files • OS platforms provide low-level APIs written in C to access these services • Problem • The diversity of hardware & operating systems makes it hard to build portable & robust networked application software • Programming directly to low-level OS APIs is tedious, error-prone, & non-portable

19. The Wrapper Facade Pattern (2/2) calls API FunctionA() calls methods Application calls API FunctionB() calls API FunctionC() Wrapper Facade void method1(){ void methodN(){ functionA(); functionA(); data functionB(); } } method1() … methodN() : Application : Wrapper : APIFunctionA : APIFunctionB Facade method() functionA() functionB() • Solution • Apply the Wrapper Facade design pattern to avoid accessing low-level operating system APIs directly This pattern encapsulates data & functions provided by existing non-OO APIs within more concise, robust, portable, maintainable, & cohesive OO class interfaces

20. ACE Socket Wrapper Facade Classes • ACE defines a set of C++ classes that address the limitations with the Socket API • Enhance type-safety • Ensure portability • Simplify common use cases • Building blocks for higher-level abstractions These classes are designed in accordance with the Wrapper Facade design pattern

21. Roles in the ACE Socket Wrapper Facade • The active connection role (ACE_SOCK_Connector) is played by a peer application that initiates a connection to a remote peer • The passive connection role (ACE_SOCK_Acceptor) is played by a peer application that accepts a connection from a remote peer & • The communication role (ACE_SOCK_Stream) is played by both peer applications to exchange data after they are connected

22. Connector/Acceptor: intent Decouples active/passive connection establishment from the service performed once the connection is established

23. The ACE_SOCK_Connector Class • Motivation • There is a confusing asymmetry in the Socket API between (1) connection roles & (2) socket modes • e.g., an application may accidentally call recv() or send() on a data-mode socket handle before it's connected • This problem can't be detected until run time since C socket handles are weakly-typed int buggy_echo_client (u_short port_num, const char *s) { int handle = socket (PF_UNIX, SOCK_DGRAM, 0); write (handle, s, strlen (s) + 1); sockaddr_in s_addr; memset (&s_addr, 0, sizeof s_addr); s_addr.sin_family = AF_INET; s_addr.sin_port = htons (port_num); connect (handle, (sockaddr *) &s_addr, sizeof s_addr); } Operations called in wrong order

24. The ACE_SOCK_Connector Class • Class Capabilities • ACE_SOCK_Connector is factory that establishes a new endpoint of communication actively & provides capabilities to • Initiate a connection with a peer acceptor & then to initialize an ACE_SOCK_Stream object after the connection is established • Initiate connections in either a blocking, nonblocking, or timed manner

25. Using the ACE_SOCK_Connector • This example shows how the ACE_SOCK_Connector can be used to connect a client application to a Web server int main (int argc, char *argv[]) { const char *server_hostname = argv[1]; ACE_SOCK_Connector connector; ACE_SOCK_Stream peer; ACE_INET_Addr peer_addr; if (peer_addr.set (80, server_hostname) == -1) return 1; else if (connector.connect (peer, peer_addr) == -1) return 1; • Instantiate the connector, data transfer, & address objects • Block until connection established or connection request failure

26. The ACE_SOCK_Acceptor Class (1/2) • Motivation • The C functions in the Socket API are weakly typed, which makes it easy to apply them incorrectly in ways that can’t be detected until run-time • The ACE_SOCK_Acceptor class ensures type errors are detected at compile-time int buggy_echo_server (u_short port_num) { sockaddr_in s_addr; int acceptor = socket (PF_UNIX, SOCK_DGRAM, 0); s_addr.sin_family = AF_INET; s_addr.sin_port = port_num; s_addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, sizeof s_addr); int handle = accept (acceptor, 0, 0); for (;;) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; write (handle, buf, n); } } Reading from wrong handle

27. The ACE_SOCK_Acceptor Class (2/2) • Class Capabilities • This class is a factory that establishes a new endpoint of communication passively & provides the following capabilities: • It accepts a connection from a peer connector & then initializes an ACE_SOCK_Stream object after the connection is established • Connections can be accepted in either a blocking, nonblocking, or timed manner • C++ traits are used to support generic programming techniques that enable the wholesale replacement of functionality via C++ parameterized types

28. Using the ACE_SOCK_Acceptor • This example shows how an ACE_SOCK_Acceptor & ACE_SOCK_Stream can be used to accept connections & send/receive data to/from a web client extern char *get_url_pathname (ACE_SOCK_Stream *); int main () { ACE_INET_Addr server_addr; ACE_SOCK_Acceptor acceptor; ACE_SOCK_Stream peer; if (server_addr.set (80) == -1) return 1; if (acceptor.open (server_addr) == -1) return 1; for (;;) { if (acceptor.accept (peer) == -1) return 1; peer.disable (ACE_NONBLOCK); // Ensure blocking <send_n>. ACE_Auto_Array_Ptr<char *> pathname (get_url_pathname (peer)); ACE_Mem_Map mapped_file (pathname.get ()); if (peer.send_n (mapped_file.addr (), mapped_file.size ()) == -1) return 1; peer.close (); } return acceptor.close () == -1 ? 1 : 0; } • Instantiate the acceptor, data transfer, & address objects • Initialize a passive mode endpoint to listen for connections on port 80 • Accept a new connection • Send the requested data • Close the connection to the sender • Stop receiving any connections

29. Reactor: intent Decouples event demultiplexing and event handler dispatching from application services performed in response to events

30. Reactor Reactor IEventHandler ConcreteHandler handleEvents() registerHandler() removeHandler() handleEvent() getHandle() handleEvent() getHandle() * Iterator handles = select(); while(handles.hasNext()) { Handle h = handles.next(); IEventHandler eh = table[h]; eh.handleEvent(); }

31. Proactor: intent Demultiplexes and dispatches service requests that are triggered by the completionof asynchronous events.

32. Proactor Proactor read() write() IEventHandler ConcreteHandler AsyncIODevice CompletionDispatcher handleEvent() handleEvent() read() write() completed() * <<create>> <<implements>>

33. Half-Sync/Half-Async: intent Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming effort without degrading execution efficiency

34. The Half-Sync/Half-Async Pattern Sync Sync Service 1 Sync Service 2 Sync Service 3 Service Layer <<read/write>> <<read/write>> Queueing Queue <<read/write>> Layer <<dequeue/enqueue>> <<interrupt>> Async Service Layer External Async Service Event Source • This pattern yields two primary benefits: • Threads can be mapped to separate CPUs to scale up server performance via multi-processing • Each thread blocks independently, which prevents a flow-controlled connection from degrading the QoS that other clients receive

35. Half-Sync/Half-Async Pattern Dynamics : External Event : Async Service : Queue : Sync Service Source notification read() work() message message notification enqueue() read() work() message • This pattern defines two service processing layers—one async & one sync—along with a queueing layer that allows services to exchange messages between the two layers • The pattern allows sync services (such as processing log records from different clients) to run concurrently, relative both to each other & to async/reactive services (such as event demultiplexing)

36. Drawbacks with Half-Sync/Half-Async Worker Thread 1 Worker Thread 2 Worker Thread 3 <<get>> <<get>> <<get>> Request Queue <<put>> handlers acceptor Event source • Problems: • Overhead when crossing inter-layer boundary • Lack of support for Async. operations in the sync. Layer • Solution: Leader/Followers pattern

37. The ACE_TSS Class (2/2) • Class Capabilities • This class implements the Thread-Specific Storage pattern, which encapsulates & enhances the native OS Thread-Specific Storage (TSS) APIs to provide the following capabilities: • It supports data that are ``physically'' thread specific, that is, private to a thread, but allows them to be accessed as though they were ``logically'' global to a program • It uses the C++ delegation operator: operator->() • It encapsulates the management of the keys associated with TSS objects • For platforms that lack adequate TSS support natively (such as VxWorks) ACE_TSS emulates TSS efficiently

38. The Thread-Specific Storage Pattern • The Thread-Specific Storage pattern allows multiple threads to use one ‘logically global’ access point to retrieve an object that is local to a thread, without incurring locking overhead on each object access manages thread 1 thread m key 1 Thread-Specific [k,t] Object accesses key n

39. Using ACE_TSS (1/3) • This example illustrates how to implement & apply ACE_TSS to our thread-per-connection logging server • In this implementation, each thread gets its own request count that resides in thread-specific storage to alleviate race conditions on the request count without requiring a mutex template <class TYPE> TYPE * ACE_TSS<TYPE>::operator-> () { if (once_ == 0) { // Ensure that we're serialized. ACE_GUARD_RETURN(ACE_Thread_Mutex, guard, keylock_, 0); if (once_ == 0) { ACE_OS::thr_keycreate(&key_); once_ = 1; } } We used the double-checked locking optimization pattern here

40. Using ACE_TSS (2/3) TYPE *ts_obj = 0; // Initialize <ts_obj> from thread-specific storage. ACE_OS::thr_getspecific (key_, (void **) &ts_obj); // Check if this method's been called in this thread. if (ts_obj == 0) { // Allocate memory off the heap and store it in a pointer. ts_obj = new TYPE; // Store the dynamically allocated pointer in TSS. ACE_OS::thr_setspecific (key_, ts_obj); } return ts_obj; }

41. Using ACE_TSS (3/3) class Request_Count { public: Request_Count (): count_ (0) {} void increment () { ++count_; } int value () const { return count_; } private: int count_; }; static ACE_TSS<Request_Count> request_count; virtual int handle_data (ACE_SOCK_Stream *) { while (logging_handler_.log_record () != -1) // Keep track of number of requests. request_count->increment (); ACE_DEBUG ((LM_DEBUG, "request_count = %d\n", request_count->value ())); } This call increments variable in thread-specific storage

42. Reminder: Using Window’s critical-section CRITICAL_SECTION cs; // Global variable void main() { if(!InitializeCriticalSection(&cs)) return; // Create threads... DeleteCriticalSection(&cs) } DWORD WINAPI ThreadProc(LPVOID lpParameter) { EnterCriticalSection(&cs); // Access the shared resource. LeaveCriticalSection(&cs); } • Now, let’s think about a CriticalSection class… • How should its interface look like?

43. The ACE Synchronization Wrapper Facades • Different operating systems provide different synchronization mechanisms with different semantics using different APIs • Some of these APIs conform to international standards, such as Pthreads • Other APIs conform to de facto standards, such as Win32 • Below we describe the following ACE classes that networked applications can use to synchronize threads and/or processes portably

44. The ACE_Lock* Pseudo-Class • The ACE mutex, readers/writer, semaphore, & file lock mechanisms all support the ACE_LOCK* interface shown below • ACE_LOCK* is a “pseudo-class,” i.e., it's not a real C++ class in ACE • We use it to illustrate the uniformity of the signatures supported by many of the ACE synchronization classes • e.g., ACE_Thread_Mutex, ACE_Process_Mutex, & ACE_Thread_Semaphore

45. The ACE_Guard Classes • Motivation • When acquiring and releasing locks explicitly, it can be hard to ensure that all paths through the code release the lock, especially when C++ exceptions are thrown • ACE provides the ACE_Guard class & its associated subclasses to help assure that locks are acquired & released properly • Class Capabilities • These classes implement the Scoped Locking idiom, which leverages the semantics of C++ class constructors & destructors to ensure a lock is acquired & released automatically upon entry to and exit from a block of C++ code, respectively

46. The Scoped Locking Idiom • Motivation • Code that shouldn’t execute concurrently must be protected by some type of lock that is acquired/released when control enters/leaves a critical section • If programmers must acquire & release locks explicitly, it is hard to ensure that the locks are released in all paths through the code • e.g., in C++ control can leave a scope due to a return, break, continue, or goto statement, as well as from an unhandled exception being propagated out of the scope void method() { lock_.acquire (); // The implementation may return prematurely… lock_.release (); } • The Scoped Locking idiom defines a guard class whose constructor automatically acquires a lock when control enters a scope & whose destructor automatically releases the lock when control leaves the scope void method() { ACE_Guard <ACE_Thread_Mutex> guard (lock_); // The lock is released when the method returns }

47. Implementing Scoped Locking in ACE template <class LOCK> class ACE_Guard { public: // Store a pointer to the lock and acquire the lock. ACE_Guard (LOCK &lock) : lock_ (&lock) { lock_->acquire ();} // Release the lock when the guard goes out of scope, ~ACE_Guard () {lock_->release (); } // Other methods omitted… private: // Pointer to the lock we’re managing. LOCK *lock_; }; Generic ACE_Guard Wrapper Facade • ACE_Write_Guard & ACE_Read_Guard acquire write locks & read locks, respectively • Instances of the ACE_Guard<T> classes can be allocated on the run-time stack to acquire & release locks in method or block scopes that define critical sections

48. ACE Condition Variable Classes (1/2) • Motivation • Condition variables allow threads to coordinate & schedule their processing efficiently • Condition variables are more appropriate than mutexes or semaphores when complex condition expressions or scheduling behaviors are needed • e.g., condition variables are often used to implement synchronized message queues that provide “producer/consumer” communication to pass messages between threads Request Queue Producer Thread Consumer Thread <<get>> <<put>> put() get() uses uses 2 ACE_Thread_Condition ACE_Thread_Mutex wait() signal() broadcast() acquire() release()

49. ACE Condition Variable Classes (2/2) • Class Capabilities • The ACE_Condition_Thread_Mutex uses the Wrapper Façade pattern to guide its encapsulation of process-scoped condition variable semantics • The ACE_Null_Condition is a zero-cost class whose interface conforms to the ACE_Condition_Thread_Mutex