1 / 11

Self-Tuning Memory Management of A Database System

Self-Tuning Memory Management of A Database System. Yixin Diao diao@us.ibm.com. Memory pools. DB2 Self-Tuning Memory Management. DB2 UDB Server. Technical problems Large systems with varying workloads and many configuration parameters Autonomic computing: systems self-management.

valentine-gaines
Download Presentation

Self-Tuning Memory Management of A Database System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Self-Tuning Memory Management of A Database System Yixin Diao diao@us.ibm.com Sigmetrics 2008 Tutorial: Introduction to Control Theory and Its Application to Computing Systems

  2. Memory pools DB2 Self-Tuning Memory Management DB2 UDB Server • Technical problems • Large systems with varying workloads and many configuration parameters • Autonomic computing: systems self-management Memory pools DB2 Clients Agents Disks • Challenges from systems aspects • Heterogeneous memory pools • Dissimilar usage characteristics • Challenges from control aspects • Adaptation and self-design • Reliability and robustness SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  3. Measured Output 1 Resource Allocation 1 Resource Consumer 1 Load Balancer 0.16 Resource Consumer N Saved System Time (xi ) Resource Allocation N 0.14 BenefitPerPage (yi ) 0.12 Measured Output N Resource 0.1 savedTime 0.08 OLTP simPages 0.06 0.04 Benefit (sec/page) 0.02 0 0 1000 2000 3000 4000 5000 Entry size (Page) Memory Pool Size (ui ) Load Balancing for Database Memory Load Balancing • Fairness  optimal ? • Common measured output ? SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  4. Regulatory Control Karush-Kuhn-Tucker conditions Constrained Optimization Constrained Optimization and Regulatory Control Saved Disk Time ( xi ) Saved System Time (xi ) Overall Mem pool 1 (x1) BenefitPerPage (y1) Mem pool 2 (x2) Mem size 1 (u1) MemoryPool1 Mem size 2 (u2) Optimal memory allocation SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  5. Dynamic State Feedback Controller • State space model • Control error • Integral control error • Feedback control law SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  6. Linear quadratic regulation (LQR) J =  [eT(k) eTI(k)] Q [eT(k) eTI(k)]T + uT(k) R u(k) Define Q and R regarding to performance Benefit Pool Size Ts=12449 Ts=15703 Ts=24827 Incorporating Const of Control into Controller Design Remove these pages Memory Pool A before Disk write dirty pages to disk after Memory Pool B before OS allocate extra memory Major cost: write dirty, move memory, victimize hot • Cost of transient load imbalances • Cost of changing resource allocations SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  7. Response Time Benefit Interval Tuner DB2 Clients Step Tuner Entry Size MIMO Control Algorithm Y Memory Statistics Collector Greedy (Constraint) Model Builder Accurate N Fixed Step 4-Bit (Oscillation) Entry Size DB2 Memory Pool Response Time Benefit Adaptive Controller Design Local linear model Decentralized integral control SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  8. 7000 6000 • DSS workload: various query lengths Time in seconds 5000 • DSS workload: index drop ConfigAdvisor settings Ts = 26342s 4000 STMM tuning Ts = 10680s Execution time for Query 21 (10 stream avg) 3000 avg = 6206 Reduce 63% 2000 Some indexes dropped avg = 2285 1000 > 2x improvement avg = 959 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Order of execution squid.torolab.ibm.com Machine: IBM7028-6C4 CPU: 4x 1453MHz Memory: 16GB Disk: 25x 9.1G Experimental Assessment • OLTP workload: multiple (20) buffer pools Response time benefits Memory sizes Increase TP from ~100 to ~250 Throughput SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  9. Comparing Control and Optimization Techniques Control-based approach Optimization-based approach Local linear model Gradient method Decentralized integral control Projected gradient (quasi-Newton) Step length (modified Armijo rule) Constraint enforcement (projection method) Strictly applies constrained optimization Less dependence on the model Similarity in a simplified scenario Differences in design considerations • “Pure” average vs. convex sum • Pole location vs. Armijo rule • Steady-state gain vs. Hessian matrix SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  10. Effect of noise (multiple runs) More robust and better uncertainty management Faster convergence, but more sensitive to noise Simulation Study: Comparison with Optimization Approach Control-based approach Optimization-based approach Memory size WL change Without noise (single run) Total saved time Control intervals SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

  11. Summary • DB2 self-tuning memory management • Interconnection, heterogeneity, adaptation and robustness, cost of control • Constrained optimization with a linear feedback controller • Experimental assessment for OLTP and DSS workloads SIGMETRICS 2008: Introduction to Control Theory. Abdelzaher, Diao, Hellerstein, Lu, and Zhu.

More Related