1 / 20

Flexible and Efficient Control of Data Transfers for Loosely Coupled Components

Flexible and Efficient Control of Data Transfers for Loosely Coupled Components. Joe Shang-Chieh Wu http://meou.us Department of Computer Science University of Maryland, USA. Obtain more accurate results by coupling existing (parallel) physical simulation components

idalia
Download Presentation

Flexible and Efficient Control of Data Transfers for Loosely Coupled Components

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Flexible and Efficient Control of Data Transfers for Loosely Coupled Components Joe Shang-Chieh Wu http://meou.us Department of Computer Science University of Maryland, USA

  2. Obtain more accurate results by coupling existing (parallel) physical simulation components Different time and space scales for data produced in shared or overlapped regions Runtime decisions for which time-stamped data objects should be exchanged Performance might be a concern What & How

  3. Approximate Match [Grid 2004] Collective Buffering [IPDPS 2007] Distributed App Match + Eager Transfer [under submission] Conclusion Roadmap

  4. Separate matching (coupling) information from the participating components Maintainability – Components can be developed/upgraded individually Flexibility – Change participants/components easily Functionality – Support variable-sized time interval numerical algorithms or visualizations Matching is OUTSIDE components

  5. Arrays are distributed among multiple processes T=4 T=3 T=2 T=1 Basic Operation Importer component Exporter component Exported Distributed Array Imported Distributed Array Distributed Array Transfer Library Approximate Match Request Array for T = 2.5 Matched Array for T = 3 Runtime-based Approximate Match Library

  6. Source Sink Precision Policy Find t’ in App0, s.t. (a) t <= t’ <= t + 0.5 (b) minimize t’ – t Separate codes from matching Connection-Wise Approximate Match Configuration file Exporter App0 Importer App1

  7. Execution time is composed of Computation time (Tcomp) Buffering time (Tbuf) Matched data transfer time (Ttran) Tbuf matters when exporter components (data sources) run more slowly Ttran matters when import components (data sinks) run more slowly Dissection of Execution Time

  8. Fastest export process sends runtimematch results to slower processes in the same program Unnecessary memory copies can be avoided in slower processes Optimal State: only required exported data are buffered Collective Buffering (when exporters run more slowly)

  9. Optimal State Collective Buffering Result Copy All Copy Some Only Copy Required Data Exporting Time for the Slowest Process

  10. Bandwidth and Latency both contribute matched data transfer time Eager transfer, transferring predicted data in advance, solves bandwidth issue Distributed approximate match, running on both exporter and importer, solves latency issue Eager Transfer + Distributed Match(when importer runs more slowly)

  11. Original ET Only ET+DM

  12. Runtime-based approximate match is a solution to couple different time scale components Performance can be improved When exporter runs more slowly, avoid unnecessary memory copies When importer runs more slowly, transfer predicted data and meta-data in advance Conclusion

  13. The End

  14. Questions ?(http://meou.us)

  15. Arrays are distributed among multiple processes T=4 T=3 T=2 T=1 Basic Operation Importer component Exporter component Exported Distributed Array Imported Distributed Array Distributed Array Transfer Library Approximate Match Request Array for T = 2.5 Matched Array for T = 3 Runtime-based Approximate Match Library

  16. On-Demand Approach • Import Component Makes Request • Perform Approx Match on Export Component, and then Transfer Matched Data • Need Data Transfer Time (T3 – T2) and 2 one-way delays (T2 – T1)

  17. Eager Transfer Only • Get permission to push predicted data • Transfer predicted data in advance • Import component makes request • Perform approx match on export component • Need 2 one-way delays ( T16 – T15)

  18. Eager Transfer With Distributed Match • … • Transfer predicted data + meta-data in advance • Import component makes request becomes local operations • Local operation time T26 – T25 is needed, independent to one-way delay

  19. All Together

  20. <importer request, exporter matched, desired precision> = <x, f(x), p> LUB minimum f(x) with f(x) ≥ x GLB maximum f(x) with f(x) ≤ x REG f(x) minimizes |f(x)-x| with |f(x)-x| ≤ p REGU f(x) minimizes f(x)-x with 0 ≤ f(x)-x ≤ p REGL f(x) minimizes x-f(x) with 0 ≤ x-f(x) ≤ p FASTR any f(x) with |f(x)-x| ≤ p FASTU any f(x) with 0 ≤ f(x)-x ≤ p FASTL any f(x) with 0 ≤ x-f(x) ≤ p Supported matching policies

More Related