1 / 54

Sizing RMS Global Buffers

Sizing RMS Global Buffers. Keith Parris Systems/Software Engineer June 18, 2008 Session 1664. RMS Buffers. RMS Buffers. RMS buffers hold buckets from RMS files Buckets are contiguous chunks of blocks A bucket might contain index info or data records

joel-garner
Download Presentation

Sizing RMS Global Buffers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sizing RMS Global Buffers Keith Parris Systems/Software Engineer June 18, 2008 Session 1664

  2. RMS Buffers

  3. RMS Buffers • RMS buffers hold buckets from RMS files • Buckets are contiguous chunks of blocks • A bucket might contain index info or data records • Note the difference between RMS Local and Global Buffers: • Local: Not shared – local to a process. • P1 process address space • Global: Shared among processes on a node. • P2 address space as of 8.3; P0 address space before

  4. Benefits of RMS Global Buffers • Processes on a node can share Global Buffers, instead of each one having to hold its own copy of a bucket in a Local Buffer • can save memory overall • allows processes to avoid reading updated buckets from disk • can improve buffer cache hit rates • New RMS algorithm introduced in OpenVMS version 7.2-1H1 and above reduced lock request rates when an RMS Global Buffer is used instead of a Local Buffer

  5. Internal Structure of an RMS Indexed File

  6. RMS Data Bucket Contents Data Bucket Data Record Data Record Data Record Data Record Data Record Data Record Data Record Data Record Data Record Data Record

  7. Selecting Candidate Files for RMS Global Buffers

  8. Approaches to Selecting Candidate Files for RMS Global Buffers • Aim to reduce I/O Rates • and improve I/O response times • Aim to reduce Lock Rates • and reduce CPU interrupt overhead and reduce cluster interconnect loading and improve lock request response times

  9. Selecting Candidate Files for RMS Global Buffers • I/O Rates

  10. Identifying files on which Global Buffers can improve I/O Performance • Focus first on disk volumes with high I/O rates. Tools: • T4 • SDA> XFC SHOW VOLUME/BRIEF • HP PerfDat, or ECP • Availability Manager • Look for hot files. Tools: • XFC: • $SHOW MEMORY/CACHE=(VOLUME=x,TOPQIO) • HP PerfDat • Other performance management tools you might use: • Unicenter Performance Management for OpenVMS • PerfCap PAWZ

  11. Selecting Candidate Files for RMS Global Buffers • Lock Rates

  12. Identifying files on which Global Buffers can reduce lock rates • Focus first on files with high lock rates. Tools: • LOCK_ACTV*.COM from OpenVMS Freeware CD V6 directory [KP_LOCKTOOLS] • Gives a cluster-wide picture • SDA> LCK SHOW ACTIVE • this is per-node, so you need to do this on each node in the cluster • These show files with hottest lock rates at the top. Focus first on those files. (SYSUAF.DAT often high) • After adding global buffers to top files, do another pass and look at the new busiest lock-rate files. • Repeat this process.

  13. Sample output from LOCK_ACTV*.COM Freeware tool USRD:[PARRIS.KP_LOCKTOOLS]LOCK_ACTV_20050427_1851.RPT;1 000000204E4F4D4D4F435F4944444602000000011C7424534D52 RMS$t......FDDI_COMMON ... RMS lock tree for file [7284,1,0] on volume FDDI_COMMON File specification: DISK$FDDI_COMMON:[SYSEXE]SYSUAF.DAT;1 Total: 742 *SKYHWK 645 WLVRNE 80 EMPIRE 14 ELKTRA 2 SKYKNG 1 202020203233374148504C41762442313146 F11B$vALPHA732 Files-11 Volume Allocation lock for volume ALPHA732 Total: 65 ELKTRA 22 WLVRNE 22 *SKYKNG 21 EMPIRE 1 . . .

  14. Sample output from SDA> LCK SHOW ACTIVE $ analyze/system OpenVMS system analyzer SDA> lck show active Active Resource Tree Information (Node ELKTRA) ---------------------------------------------- RSB Address Tot Locks SubRSB Act Node Resource Name ----------------- --------- ------ ----- ------ ------------------------------- FFFFFFFD.7BC89AC0 2990 2670 179 ELKTRA F11B$vALPHA82 FFFFFFFD.7BFB35C0 0 0 21 ELKTRA OVO$MONAGTQ_ELKTRA FFFFFFFD.7BD30300 50 50 10 EMPIRE F11B$vFDDI_COMMON FFFFFFFD.7BED2F80 10 10 9 EMPIRE F11B$vUSERDISK1 FFFFFFFD.7BD39340 14 8 5 WLVRNE RMS$|......FDDI_COMMON ... File: DISK$FDDI_COMMON:[SYSEXE]RIGHTSLIST.DAT;1 FFFFFFFD.7BDFF980 4 2 3 EMPIRE RMS$t......FDDI_COMMON ... File: DISK$FDDI_COMMON:[SYSEXE]SYSUAF.DAT;1 FFFFFFFD.7BC766C0 1 1 2 ELKTRA SYS$SYS_ID94.... FFFFFFFD.7BFA9340 2 2 1 WLVRNE RMS$>..ç...ALPHA82 ... File: DISK$ALPHA82:[VMS$COMMON.SYSEXE]VMSMAIL_PROFILE.DATA;1 FFFFFFFD.7BDFF5C0 0 0 1 WLVRNE F11B$aFDDI_COMMON t... FFFFFFFD.7C69D700 2 2 1 ELKTRA RMS$.......ALPHA82 ... File: DISK$ALPHA82:[VMS$COMMON.SYSEXE]TCPIP$HOST.DAT;1 SDA>

  15. Common Approaches to Choosing Global Buffer Counts

  16. Common Approaches to Choosing Global Buffer Counts • Some of the approaches to sizing global buffer count (past and present): • Total index size • User count • Cache hit rate

  17. Common Approaches to Choosing Global Buffer Counts • Total Index Size

  18. Approaches to global buffer counts: Total Index Size Intent: Try to cache entire index • Primary Index (& sometimes Alternate indices), • plus some number of data buckets • Estimate index sizes from: • $ ANALYZE/RMS/STATISTICS output, or • $ ANALYZE/RMS/FDL/OUT=file.FDL and looking at ANALYSIS_OF_KEY sections in the output FDL file • Number of total blocks in index divided by bucket size in blocks equals number of buckets (and thus global buffers) needed to cache the entire index

  19. Example calculation based on $ANALYZE/RMS/STATISTICS output 1) Determine the bucket size of the index buckets. In this example, this is easy, since all the buckets in the entire file are 3 blocks in size. From ANALYZE/RMS/STATISTICS output: RMS FILE ATTRIBUTES ...        Bucket Size: 3 2) Determine the number of blocks in the index. To be generous, we'll count the blocks for all the indices: STATISTICS FOR KEY #0 ...        Count of Index Blocks:                 66 ... STATISTICS FOR KEY #1 ...        Count of Index Blocks:                  3 ... STATISTICS FOR KEY #2 ...        Count of Index Blocks:                  3 ...STATISTICS FOR KEY #3 ...        Count of Index Blocks:                  3 This is 66+3+3+3=75 blocks. With a bucket size of 3, this equates to 25 buckets. Add a small number for data buckets (say 5), and the number 30 seems reasonable as a result. Each global buffer holds one bucket (global buffers are sized to accommodate the size of the largest bucket in the file), so 30 global buffers would be indicated by this method.

  20. Common Approaches to Choosing Global Buffer Counts • User Count

  21. Approaches to global buffer counts: User Count Intent: Provide sufficient buckets for all users • Assume each user process needs its own data bucket to work with • plus some number of shared index buckets

  22. Common Approaches to Choosing Global Buffer Counts • RMS Global Buffer Cache Hit Rate

  23. Approaches to global buffer counts:Hit rate Intent: Raise count until cache hit rates are maximized • Enable RMS statistics-gathering first with: • $ SET FILE/STATISTICS filespec • Then use MONITOR utility to track changes in global buffer cache hit rates as count is modified over time: • $ MONITOR RMS/ITEM=CACHING/ALL/FILE=filespec • Raise global buffer count until hit rate doesn't increase any more after an increase in count. Shoot for 90-95%+ hit rate • Note: Enabling statistics and/or enabling global buffers on a file requires 1 global section per file • Watch GBLSECTIONS – it is not a dynamic parameter

  24. SYSGEN parameters involved with adding RMS Global Buffers to files

  25. Important SYSGEN parameters • Relevant SYSGEN parameters (active and dynamic): • GBLSECTIONS: static • GBLPAGES: dynamic • GBLPAGFIL: dynamic • Note: Raising GBLSECTIONS requires a reboot

  26. Effects of "bad" number choices for global buffer count • No Global Buffers

  27. Effects of "bad" global buffer count:Zero buffers Zero buffers: • Obviously no global caching; all local buffers • duplicated content for each process • multiple reads to obtain multiple copies • buffer swapping/updates via disk drive • Increase in locking activity without global buffers • Better locking algorithms for global buffers as of 7.2-1H1 or above • Lower buffer hit rates

  28. Effects of "bad" number choices for global buffer count • Too-Few Global Buffers

  29. Effects of "bad" global buffer count:Too low Too low: • More I/Os require use of Local RMS buffers instead of Global RMS buffers • Lower buffer hit rates • Global, because of fewer Global buffers to “hit” • Local, because process may not already have a given bucket in the typically-limited set of Local buffers • Uses more memory overall, as Local RMS buffers may contain duplicated buckets which cannot be shared • More read operations from disk are required to populate these multiple copies of buckets in Local RMS buffers

  30. Effects of "bad" number choices for global buffer count • Too-Many Global Buffers

  31. Effects of "bad" global buffer count:Too high Too high: • In past versions of OpenVMS, having too many global buffers was discouraged because: • Scanning too-many global buffers could actually slow things down • P0 address space used to map global buffers might become exhausted • The number of global buffers for a file was limited to 32K maximum

  32. Previous limitations in OpenVMS on RMS global buffer counts • Scanning algorithm: • was sequential, so scanning was slow • now uses hash table (hashed lookup started in 7.2) • Global buffer address space • was in P0 space; shared with other data and limited to 1 GB • as of 8.3, can now reside in P2 (64-bit) address space • Global buffer count itself: • was 32K (signed 16-bit word) • as of 8.3, is now 2M (signed 32-bit longword)

  33. Effects of "bad" global buffer count:Too high Too high: • People tend to worry about Global Buffer memory space being "wasted“ with too high a count • Global buffer area itself is the size of the largest bucket in the file, to accommodate any bucket in the file • Memory for extra/unused global buffer headers is "wasted,“ but is not significant in size • Global Buffers reside in Paged memory, so there is no memory "loss" for unused global buffers themselves (or the unused tail end of a buffer for smaller buckets)

  34. Sizing Conclusions • There is no big downside to having "too many" global buffers now • But now we can actually see how many global buffers are actively in use, using a new technique • so we can now know when we have more global buffers than we “need”

  35. New Approach to Choosing Global Buffer Counts • Measure actual RMS Global Buffer usage in operation

  36. New Technique: See How Many Global Buffers are Actually In Use • Intent: Detect when global buffers are unused, as a hint that the count is “more than needed” • Technique: Look at Global Buffer Headers during or after heavy file activity • If buffer is presently in use, we’ll conclude it’s “needed” • If buffer has been used at any point in the past, we’ll also say it’s also “needed” • If buffer has never been used, it’s probably “extra” • Note: Some “extra” global buffers may be good, for future use

  37. New Technique: See How Many Global Buffers are Actually In Use • How do we tell if a Global Buffer is currently in use, or has ever been used? • Lock ID field starts out zero, then holds Lock ID value when used; not cleared afterward • Lock Count field being non-zero indicates when global buffer is presently being used

  38. New Technique: See How Many Global Buffers are Actually In Use: 8.3-1H1 SDA> show proc/rms=(gbdsum) Process index: 006A Name: _TNA4: Extended PID: 2CE0046A -------------------------------------------------------------------- GBD Summary ----------- GBD USE BUFFER CACHE_ RELADDR CNT SIZE NUMB VBN VBNSEQNUM RELADDR VAL LOCK_ID MODE ----------------- ---- ---- ---- --- -------- ---------------- --- -------- ---- 0000000000012480 0000 0400 0400 00000007 00000000 0000000000016400 3 6B0016EF 1 CR 0000000000012400 0000 0400 0400 00000004 0000006C 0000000000016000 0 2A0016EE 0 0000000000012380 0000 0400 0000 FFFFFFFF 00000000 0000000000015C00 0 00000000 0 0000000000012300 0000 0400 0000 FFFFFFFF 00000000 0000000000015800 0 00000000 0 0000000000012280 0000 0400 0000 FFFFFFFF 00000000 0000000000015400 0 00000000 0

  39. New Technique: See How Many Global Buffers are Actually In Use: 8.3-1H1 SDA> show proc/rms=(gbdsum) Process index: 006A Name: _TNA4: Extended PID: 2CE0046A -------------------------------------------------------------------- GBD Summary ----------- GBD USE BUFFER CACHE_ RELADDR CNT SIZE NUMB VBN VBNSEQNUM RELADDR VAL LOCK_ID MODE ----------------- ---- ---- ---- --- -------- ---------------- --- -------- ---- 0000000000012480 0000 0400 0400 00000007 00000000 0000000000016400 3 6B0016EF 1 CR 0000000000012400 0000 0400 0400 00000004 0000006C 0000000000016000 0 2A0016EE 0 0000000000012380 0000 0400 0000 FFFFFFFF 00000000 0000000000015C00 0 00000000 0 0000000000012300 0000 0400 0000 FFFFFFFF 00000000 0000000000015800 0 00000000 0 0000000000012280 0000 0400 0000 FFFFFFFF 00000000 0000000000015400 0 00000000 0 Currently in use

  40. New Technique: See How Many Global Buffers are Actually In Use: 8.3-1H1 SDA> show proc/rms=(gbdsum) Process index: 006A Name: _TNA4: Extended PID: 2CE0046A -------------------------------------------------------------------- GBD Summary ----------- GBD USE BUFFER CACHE_ RELADDR CNT SIZE NUMB VBN VBNSEQNUM RELADDR VAL LOCK_ID MODE ----------------- ---- ---- ---- --- -------- ---------------- --- -------- ---- 0000000000012480 0000 0400 0400 00000007 00000000 0000000000016400 3 6B0016EF 1 CR 0000000000012400 0000 0400 0400 00000004 0000006C 0000000000016000 0 2A0016EE 0 0000000000012380 0000 0400 0000 FFFFFFFF 00000000 0000000000015C00 0 00000000 0 0000000000012300 0000 0400 0000 FFFFFFFF 00000000 0000000000015800 0 00000000 0 0000000000012280 0000 0400 0000 FFFFFFFF 00000000 0000000000015400 0 00000000 0 Formerly in use

  41. New Technique: See How Many Global Buffers are Actually In Use: 8.3-1H1 SDA> show proc/rms=(gbdsum) Process index: 006A Name: _TNA4: Extended PID: 2CE0046A -------------------------------------------------------------------- GBD Summary ----------- GBD USE BUFFER CACHE_ RELADDR CNT SIZE NUMB VBN VBNSEQNUM RELADDR VAL LOCK_ID MODE ----------------- ---- ---- ---- --- -------- ---------------- --- -------- ---- 0000000000012480 0000 0400 0400 00000007 00000000 0000000000016400 3 6B0016EF 1 CR 0000000000012400 0000 0400 0400 00000004 0000006C 0000000000016000 0 2A0016EE 0 0000000000012380 0000 0400 0000 FFFFFFFF 00000000 0000000000015C00 0 00000000 0 0000000000012300 0000 0400 0000 FFFFFFFF 00000000 0000000000015800 0 00000000 0 0000000000012280 0000 0400 0000 FFFFFFFF 00000000 0000000000015400 0 00000000 0 Never used

  42. New Technique: See How Many Global Buffers are Actually In Use: Pre-8.3 SDA> show proc/rms=(gbdsum) Process index: 0038 Name: FSS001 Extended PID: 312E7838 -------------------------------------------------------------------- GBD Summary ----------- GBD USE REL_ CACHE_ Address CNT SIZE NUMB VBN VBNSEQNUM ADDR VAL LOCK_ID MODE ------- ----- ---- ---- --- ------- ---- --- ------- ----- 14266780 0000 0400 0400 0000002B 00000001 00021C00 0 71034790 1 CR 14266280 0000 0400 0400 00000057 00000000 0001F400 0 6F05320A 1 CR 14266A80 0000 0400 0400 0000002D 00000000 00023400 0 4A05DB79 1 CR ... 14266D80 0000 0400 0400 00000029 00000000 00024C00 0 4B04A2B8 0 ... 14265780 0000 0400 0000 FFFFFFFF 00000000 00019C00 0 00000000 0 14265700 0000 0400 0000 FFFFFFFF 00000000 00019800 0 00000000 0 14265680 0000 0400 0000 FFFFFFFF 00000000 00019400 0 00000000 0 Currently in use Formerly in use Never used

  43. Example DCL Procedure • Sample DCL command procedure is available to check RMS Global Buffer usage using this technique: • GLOBAL_BUFFER_USAGE.COM • Available at http://www2.openvms.org/kparris/ • Generates a .CSV file with filename and Current, Peak, and Total RMS Global Buffer counts for each open file on the system where there are RMS Global Buffers in use • Measure RMS Global Buffers usage for open files for all processes on the node on which it is run. Implications: • May need to run it on each node to get a cluster-wide picture • If files get closed, information on RMS Global Buffer usage for those files would be lost

  44. Example DCL Procedure Usage: $ @GLOBAL_BUFFER_USAGE.COM Results may be found in the file GLOBAL_BUFFER_USAGE_1-APR-2008_HASBRO.CSV $ type GLOBAL_BUFFER_USAGE_1-APR-2008_HASBRO.CSV "File Specification","Current","Peak","Total" "HASBRO$DKB600:[VMS$COMMON.SYSEXE]SYSUAF.DAT;1",139,140,172 $ When loaded into Excel, it looks something like this:

  45. Update on OpenVMS Version 8.3 Changes for Global Buffer Sizing

  46. Update on Version 8.3 changes • New DCL options for SET FILE/GLOBAL_BUFFER • /GLOBAL_BUFFER[=keyword[=n]] • /NOGLOBAL_BUFFER • The keyword can be: • COUNT=n - The value n sets the longword count of the number of global buffers • PERCENT=p - The value p expresses the size of the global cache as a percent of the total number of blocks currently used in the file. • DEFAULT - Requests RMS at runtime to recalculate the global cache size based on an algorithm that makes use of two new global buffer SYSGEN parameters: GB_CACHEALLMAX and GB_DEFPERCENT

  47. Update on Version 8.3 changes • Two settings (potentially) in the file header now: pre-8.3 and 8.3+ • Pre-8.3 nodes will use old fixed 16-bit value field • 8.3+ nodes may have more global buffers, and new Flags field indicates whether to use the larger 32-bit fixed value COUNT field or new calculated values under the PERCENT or DEFAULT options

  48. Update on Version 8.3 changes Examples: • Fixed count values: • Pre-8.3: • $ SET FILE/GLOBAL_BUFFER=1000 filespec • 8.3+: • $ SET FILE/GLOBAL_BUFFER=COUNT=1000000 filespec • New PERCENT option for 8.3+: • $ SET FILE/GLOBAL_BUFFER=PERCENT=35 filespec • New DEFAULT option for 8.3+: • $ SET FILE/GLOBAL_BUFFER=DEFAULT filespec

  49. Update on Version 8.3 changes $ DIRECTORY/FULL output: • Pre-8.3 fixed-count setting: Global buffer count: 1000 • 8.3+ indicates old count plus new options: Global buffer count pre-V8.3: 1000 Global buffer count V8.3+ : 35 percent $ ANALYZE/RMS output: Global Buffer Count pre-V8.3: 1000 Global Buffer Count V8.3+ : 35 Global Buffer Flags V8.3+ : percent

  50. Update on Version 8.3 changes SYSGEN Help for GB_CACHEALLMAX: Sys_Parameters GB_CACHEALLMAX (Alpha and I64) If a file is connected to RMS with the RMS global buffer DEFAULT option enabled, the number of blocks cached is either a maximum of the GB_CACHEALLMAX parameter or a percentage of the file, whichever results in a larger global count. Note that although a maximum cache size of %x7FFFFFFF is supported for an indexed file, sequential and relative file organizations are restricted to a maximum cache size of 32767. GB_CACHEALLMAX is a DYNAMIC parameter.

More Related