1 / 9

Resource Management in the Greenplum Parallel Database

Siva Narayanan (siva@greenplum.com) Consultant Software Engineer, Query Processing, EMC Greenplum. Resource Management in the Greenplum Parallel Database. Why, you ask?. Problem. Finite resources - CPU/memory/IO/network Concurrent activity Different business value (Loads/Reports/Analytics)

aron
Download Presentation

Resource Management in the Greenplum Parallel Database

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Siva Narayanan (siva@greenplum.com) Consultant Software Engineer, Query Processing, EMC Greenplum Resource Management in the Greenplum Parallel Database

  2. Why, you ask?

  3. Problem • Finite resources - CPU/memory/IO/network • Concurrent activity • Different business value (Loads/Reports/Analytics) • Different system impact (Simple/Complex queries) • How can a DBA manage the system and keep everyone happy?

  4. Solution • Determine business value of a query upon arrival • Translate that to fair share of CPU and Memory • Resource reservation / Admission control • Are the resources available? • Run-time resource allocation • Ensure that reservations are honored • Adjust behavior as necessary

  5. CPU vs Memory

  6. CPU Sharing • Every query operator in a execution plan • Continually measures its actual CPU usage and compares it with fair share • If it uses too much, it sleeps for a short while • Rinse, repeat • I/O and network bandwidths are similar

  7. Memory Sharing • Every query operator in a execution plan • Gets a portion of memory reserved for the entire query • Memory intensive operators vs not • Re-use memory between blocking operators • If data is too large, they spill • Net effect, every query uses up to its fair share

  8. Summary • Resource management is a big problem with big data • Align resource allocation with business value • Greenplum Parallel Database has mechanisms for CPU and Memory

  9. Questions? We’re hiring! siva@greenplum.com

More Related