1 / 14

A User-Lever Concurrency Manager

A User-Lever Concurrency Manager. Hongsheng Lu & Kai Xiao. Motivation. Modern Computers with multiple resources Multi-core processor, large memory, many disks, etc. Given a batch of independent programs and commands command 1: cp a.tar.gz b. tar.gz

muhammad-jasmi
Download Presentation

A User-Lever Concurrency Manager

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A User-Lever Concurrency Manager Hongsheng Lu & Kai Xiao

  2. Motivation • Modern Computers with multiple resources • Multi-core processor, large memory, many disks, etc. • Given a batch of independent programs and commands • command 1: cp a.tar.gz b. tar.gz • command 2: mc_dose_calculation inputdata.txt • command 3: … et al. • Can we guarantee to use full horsepower of multiple resources in order to finish them as soon as possible? • Weapon: Concurrency

  3. Problem • Physical concurrency makes effective parallelism and improves the performance • However, over loading concurrency level would actually deteriorate the system running performance • Low: wasting computing horsepower • High: performance comes down • Mission of our controller: dynamically adjust the concurrency level. • Requirement: monitoring and controlling • First step: understanding system behavior under different concurrency level.

  4. Observation: CPU bound • CPU bound test : Simulation program Observation: 99% CPU for user Observation: Swap-in and Swap-out begin If increase the workload of application

  5. Observation: IO bound • IO bound test: Write 50MB data to disk per program (30 total) • Extreme test: Write 1GB data to disk per program (90 total) Writing speed < Flushing speed Processes blocked Writing speed >> Flushing speed Too Many process blocked Writing speed >= Flushing speed Processes blocked n=1 n=90 Running time = 4.5 s Running time = 4.0 s Running time = 189 s Running time = 220 s

  6. Solution Feedback & Record(FR) Block Not Kill What kind of processes should be blocked?

  7. Solution Block Not Kill

  8. Solution Block Not Kill 1. If there are procs blocked, don’t fork new procs. 2. Blocked procs are activated after some time one by one 3. If either resource is under usage, activate one of appropriate blocked procs immediately

  9. Benchmark Set B1 : Only CPU procs with flat memory requirement B1': Only CPU procs with increase memory requirement B2 : IO procs with few CPU procs of flat memory requirement B2': IO procs with few CPU procs of increase memory requirement B3 : Random procs with flat memory requirement (Short length) B3’: Random procs with increase memory requirement (Short length) B4 : Random procs with flat memory requirement (Medium length) B4': Random procs with increase memory requirement(Medium length) B5 : Random procs with flat memory requirement (Large length) B5': Random procs with increase memory requirement (Large length)

  10. Evaluation

  11. Conclusion 1. Record accuracy FR Performance

  12. Conclusion 1. Record FR 2. Memory FR with right records behaviors good When benchmark needs increasing memory, FR with right record is better than Best N While benchmark with flat memory requirement, FR is almost equal to best N

  13. Conclusion 1. Record FR 2. Memory FR with right records behavior good With increasing memory, the improvement is more obvious when job is longer

  14. Conclusion 1. Record FR 2. Memory FR with right records behavior good 2. Job length Improvement is more obvious Questions? Thank you !

More Related