1 / 21

Managing the Performance Impact of Administrative Utilities

Managing the Performance Impact of Administrative Utilities. Paper by S. Parekh ,K. Rose , J.Hellerstein , S. Lightstone, M.Huras, and V. Chang Presentation and Discussion Led by N. Tchervenski CS 848, University of Waterloo November 1, 2006. Outline.

amelia
Download Presentation

Managing the Performance Impact of Administrative Utilities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Managing the Performance Impact of Administrative Utilities Paper by S. Parekh ,K. Rose , J.Hellerstein , S. Lightstone, M.Huras, and V. Chang Presentation and Discussion Led by N. Tchervenski CS 848, University of Waterloo November 1, 2006

  2. Outline • Introduction – performance impact of administrative utilities • Proposed solution • Architecture and Control Theory • Tests performed • Conclusion • Discussion

  3. Performance Impact of Administrative Utilities • Administrative utilities • Essential to the system • Have performance impact • With 24/7 operation, it is never a good time to suffer performance degradation • Solution: find a way to slow down

  4. Example of DB Running a Backup * Throughput and response time averaged over 60s intervals

  5. How to Slowdown a Utility • Performance impact is dynamic – both for utilities and regular workloads (WLs) • Low level approach • per-resource quotas / priorities • difficult to manage • Admin Utility Performance Policy - at most x% degradation of production work • How to throttle utilities  SIS – self-imposed sleep • How to translate policy requirement vs. throttling units?

  6. SIS – Self-imposed Sleep

  7. Action Interval and Sleep Fraction • Action interval = workTime + sleepTime • With action interval being constant, we need just sleep fraction: • Sleep fraction = sleepTime / action interval • Sleep fraction = 0  unthrottled, 1  stopped • Suggested value for action interval is at least a few iterations of the “main-loop” of the utility

  8. Throttle Manager Architecture Action interval = const PI controller X% sleepTime Linear model based on <sleepTime , performance>

  9. Degradation Estimator • Baseline estimator – system performance w/o utilities • Degradation = 1 – performance / baseline • How to determine baseline? • Stop all utilities  WL surges, short-term performance, underutilize resources • Linear fitting of <sleepTime, performance> • Performance = f(sleepTime) = Q1*sleepTime+Q0 • Recursive least squares and exponential forgetting

  10. Linear Fit Example of Sleep/Throughput Actual baseline Estimated baseline Steady workload, backup throttling kept constant for 20 minute intervals

  11. Controller • Goal: current degradation = degradation limit • Error = degradation limit – current degradation • PI controller used • Throttling(k+1) = Kp * error(k) + Ki * Sum(error(i), i=0..k) • Kp – proportional gain – used to increase speed of response • Ki – integral gain – eliminate steady state error • Kp, Ki and control interval can be hard-coded or determined at runtime • Kp and Ki can be estimated by utilizing pole placement from control theory, but experimental results are necessary to confirm results [2] • Experiments in this paper: • control interval = 20 seconds • Kp and Ki same across all experiments

  12. Tests Performed • Testbed description • DB2 v8.1, 4-CPU RS/6000, 2GB ram, AIX 4.3, 8 physical disks • Workload similar to TPC-C • Initial “warm-up” period of 10 minutes, to stabilize system / bufferpools /etc. • Utility used – parallelized BACKUP – multiple processes reading from multiple tablespaces, and multiple other processes writing to separate disks

  13. OS Priorities vs SIS (Sleep fraction) Linear effect when throttling using sleep No performance gain by changing OS priority of backup process WL alone 100% throttling OS priority works for CPU intensive WLs, here we have I/O intensive WL. CPU is idle 80% of the time.

  14. Dynamic Effect of SIS. Does “Turning the Knob” Actually Do Something? 15tps avg Backup started As in previous slide, we don’t get back to 100% throughput when fully throttled, but we’re close.

  15. Feedback Control X=30% degradation policy

  16. Feedback Control Effectiveness • Without BACKUP – 15tps • With x=30%, steady workload – 25 users •  9.4tps  38% degradation • Why the throttling slump? • Throttling system compensates for decreasing resource demands of the backup? • With x=30%, Workload surge at 1500s – from 10 users to 25 users. • Pre-surge degradation of 36% • Post-surge degradation of 19% • Still good results, close to the 30% policy

  17. Causes for Deviation • Baseline estimator – actual throughput is 15.1 tps vs projected value of 13.2tps.. • System stochastics not always estimate degradation correctly • For example, the drop of throttle at t=1800s • Quick to self-correct  correct results in the long term • Short-term violations could be avoided by trading adaptation speed by adjusting the forgetting factor in online estimator.

  18. Conclusion • Administrative utilities must be run, but there is no timeslot for them • Proposed an application-based throttling mechanism – need to change applications code only, but OS/system independent • Easy for administrators to just specify degradation policy • Applicable to various systems • Main requirements • Utility work be identifiable – put sleep there • Performance can be measured and w/o much overhead

  19. Limitations and Future Work • Test on multiple utilities • Throttle each utility separately? • Propose and analyze different approaches for the controller • PI algorithm, recursive least squares estimator, etc. How to specify parameters for them? • Automate determination of controller parameters as they are system dependent.

  20. Discussion • Why the throttling slump on the feedback control? • Even when backup is fully throttled, system may not reach peak performance as before, since it needs more time to stabilize (i.e. bufferpools again). This may be a better explanation for the difference between projected baseline and the actual baseline. • Even if tasks were CPU intensive, assigning them priority by OS is not guaranteed to work, since they may interact with other parts of the engine – issue queries, etc.. Can’t slow the engine for that. • Obviously this works since it’s been implemented in DB2 v8 and v9 – backup / rebalance / auto-runstats – all I/O intensive tasks. • Other ways to limit/control the impact of backup to DB system. Controlling bufferpools / memory. Automatic tuning of memory is introduced in DB2 v9. • How to handle peak loads? How do we guarantee QoS? • Can we monitor not only TPS output, but try to “expect” what the WL performance would be, based on # of clients , # of queries compiled/executed, bufferpool activity/misses?

  21. References [1] Sujay Parekh, Kevin Rose, Joseph L. Hellerstein, Sam Lightstone, Matthew Huras, and Victor Chang. Managing the performance impact of administrative utilities. In Self-Managing Distributed Systems - 14th IFIP/IEEE International Workshop on Distributed Systems: Operations and Management (DSOM 2003), number 2867 in Lecture Notes in Computer Science. Springer-Verlag, 2003. [2] Diao,Y,Gandhi,N.,Hellerstein,J.L.,Parekh,S.,Tilbury,D.M.:Using MIMO feedback control to enforce policies for interrelated metrics with application to the Apache web server. In: Proceedings of Network Operations and Management. (2002)

More Related