1 / 41

Reducing Interprocess Communication Overhead in Concurrent Programs

Reducing Interprocess Communication Overhead in Concurrent Programs. Erik Stenman Kostis Sagonas. Motivation. Concurrency as an abstraction is important. Systems that need to interact with the outside world are hard to model without concurrency. Unfortunately concurrency costs.

zinna
Download Presentation

Reducing Interprocess Communication Overhead in Concurrent Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reducing Interprocess Communication Overhead in Concurrent Programs Erik Stenman Kostis Sagonas

  2. Motivation • Concurrency as an abstraction is important. • Systems that need to interact with the outside world are hard to model without concurrency. • Unfortunately concurrency costs. • There are two types of runtime overheads: • ”Direct overhead” of concurrency primitives. • ”Indirect overhead” from hiding the data-flow from the optimizing compiler.

  3. Goal of this work • Reduce the overhead of concurrency in concurrent programs. Idea • Optimize the code that implements process communication.We call this interprocess optimization, and we will present three techniques: • Rescheduling send. • Direct dispatch send • Interprocess inlining.

  4. Rescheduling Send • Typically, when a process sends a message, it is because it wants the receiver to act on that message. • It is therefore in the interest of the sender to yield to the receiver and allow it to promptly act on the sent message. • We call this type of send, a rescheduling send.

  5. Rescheduling Send Implementation: • The send operation suspends the currently active (sending) process. Benefits: • lower message passing latency. • better cache behavior (the receiver has access to the message while it is still hot in the cache).

  6. Direct Dispatch Send • The sender contributes the remaining part of its time-slice to the receiver (hoping this would lead to a faster response). • The receiver then is woken up directly (ignoring the ready queue). • Overhead of the scheduler is eliminated. • If the receiver also performs a direct dispatch send back to the sender, interprocess communication becomes as fast as a function call.

  7. Interprocess Inlining • Merge code from the sender with code from the receiver.

  8. A message Processa The sender Process b The receiver

  9. Known communication protocol.Can be optimized.Process b only needs to be ready to receive the communication.

  10. The state of process b has changed. Without really participating in the actual communication.

  11. Interprocess Inlining • Merge code from the sender with code from the receiver. • In the merged code, the overhead of the communication protocol can be optimized away. • We suggest using profiling to find situations where this is feasible. • This requires that the optimized code is guarded by a runtime test.

  12. ref_server(V) -> receive {‘++’,From}-> From ! {self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From ! {self(),NewV}, ref_server(NewV); {‘=’,From}-> From ! {self(),V}, ref_server(V) end inc(Beta) -> Beta ! {‘++’,self()}, receive {Beta,Answer} -> Answer; end. An example in Erlang…

  13. ref_server(V) -> receive {‘++’,From}-> From ! {self(),V+1}, ref_server(V+1); {‘:=’,From,NewV}-> From ! {self(),NewV}, ref_server(NewV); {‘=’,From}-> From ! {self(),V}, ref_server(V) end inc´(Beta) -> ifready(Beta)-> B_V=get(‘V’,Beta)+1, save(‘V’,B_V,Beta), B_V; true -> inc(Beta) end. …could be rewritten as

  14. Ready Code merging: the general case  g ’  - Head g - Head  - Head Createmessage Createmessage Receive Copy Send g – tail Yes Restoreβ-State -Tail Send Extractedg – tail -Tail Saveβ-State Copy of-Tail

  15. Code merging Can only be done if: • Code explosion does not occur. • The code does not suspend. • The control flow does not escape the included code. • The extracted code terminates (o/w, the sending process might hang after code merging.)

  16. Code merging • These instructions are not extracted: • A call or a return. • Instructions that lead to the suspension of the process. • Throwing an exception. • Some BIFs are large and uncommon and not worth the effort to adapt for external execution. • Non-terminating code is unacceptable. • Hence, we do not accept any loops in the control flow graph.

  17. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f(Beta) -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end. Createmessage send tail Code merging

  18. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f(Beta) -> Mess = {hi,self()}, ifready(Beta)-> NEWCODE; true -> Beta ! Mess, receive {Beta,fine} -> right; {Beta,_Answer} -> wrong end end. Test f-tail Code merging

  19. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: b-Mbox = [{hi,self()}], caseb-Mboxof [{hi, b-From}]-> b-From !b-{b-self(),fine}; _-> b-ignore b-end, save_state(Beta), COPY-OF-f-TAIL Code merging

  20. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: b-Mbox = [{hi,self()}], case [{hi,self()}]of [{hi, b-From}]-> b-From !b-{b-self(),fine}; _-> b-ignore b-end, save_state(Beta), COPY-OF-f-TAIL Code merging

  21. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: b-Mbox = [{hi,self()}], b-From = self(), b-From !b-{b-self(),fine}, save_state(Beta), COPY-OF-f-TAIL Code merging

  22. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: b-Mbox = [{hi,self()}], b-From = self(), self()!b-{b-self(),fine}, save_state(Beta), COPY-OF-f-TAIL Code merging

  23. Return messages • An important special case • Handled by applying the same type of rewriting. • The ready test is replaced by a test that checks that both mailboxes are empty.

  24. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: b-Mbox = [{hi,self()}], b-From = self(), self()!b-{b-self(),fine}, save_state(Beta), COPY-OF-f-TAIL More Code merging

  25. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: b-Mbox = [{hi,self()}], b-From = self(), a-Mbox = {b-self(),fine}, save_state(Beta), COPY-OF-f-TAIL More Code merging

  26. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: a-Mbox = {b-self(),fine}, save_state(Beta), COPY-OF-f-TAIL More Code merging

  27. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: a-Mbox = {b-self(),fine}, save_state(Beta), case a-Mbox of {Beta,fine} -> right; {Beta,_Other} -> wrong end More Code merging

  28. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: a-Mbox = {b-self(),fine}, save_state(Beta), case {b-self(),fine} of {Beta,fine} -> right; {Beta,_Other} -> wrong end More Code merging

  29. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: a-Mbox = {b-self(),fine}, save_state(Beta), right More Code merging

  30. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). NEWCODE: save_state(Beta), right More Code merging

  31. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). More Code merging f´(Beta) -> ifready(Beta)->save_state(Beta), right; true -> f(Beta) end.

  32. This optimization gives • Access to variables in another process with almost no overhead. (Two reads, one test, and two writes.) • The overhead of the communication protocol can be reduced. (Creating a tuple and switching on it.)

  33. Profiling • We have profiled parts of Eddie and the AXD code and found: • That each send goes to one particular receive. • The receiving process is typically suspended with an empty mailbox. • The same profiling tool could be used in a dynamic compiler to find pairs of senders/receivers whose inter-process communication can be optimized.

  34. Conclusions • Presented several different methods for cross-process optimization that reduce the overhead of interprocess communication. • No modifications to existing code are required. • Open issue is their integration into the Erlang development environment

  35. That’s all folks! Praeterea censeo "0xCA" scribere Erlang posse.Happi

  36. Problem • How does one find senders and receivers that communicate? • Static analysis • Problematic in a language with dynamic typing, dynamic process creation, communication through mailboxes, and asynchronous communication. • Profiling & dynamic recompilation

  37. Interprocess inlining • Do: • Find two communicating processes. • Merge code from the sender with code from the receiver. • Optimize the merged code. • Get: • Reduced message passing. • Short-circuited code for switching on messages.

  38. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f(Beta) -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end. An example in Erlang…

  39. g() -> receive {hi,From}-> From ! {self(),fine}; _ -> ignore end, g(). f´(Beta) -> ifready(Beta)->save_state(Beta), right; true -> Beta ! {hi,self()}, receive {Beta,fine} -> right; {Beta,_Other} -> wrong end end. …could be rewritten as

More Related