1 / 51

Linux And The Progress RDBMS

Linux And The Progress RDBMS. Gus Björklund (gus@progress.com) Wizard Progress Software. PUG Challenge 2002, Veldhoven, the Netherlands. PROGRESS S O F T W A R E. Engine Crew. Abstract.

erv
Download Presentation

Linux And The Progress RDBMS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Linux And The Progress RDBMS Gus Björklund (gus@progress.com) Wizard Progress Software PUG Challenge 2002, Veldhoven, the Netherlands

  2. PROGRESS S O F T W A R E Engine Crew

  3. Abstract The Linux operating system can be the right choice for many Progress RDBMS installations. This talk describes how an Intel-based system can be a highly cost-effective and powerful database server running the Progress RDBMS on RedHat Linux 7.2 on a powerful but inexpensive computer. We will show how to configure the system, what to do, and some things to avoid. Informal benchmark results from the Spring of 2002 are used to illustrate the performance of various configurations. Time: 90 minutes

  4. Please ask questionsif I do not explain something clearly

  5. Spring 2002Linux Benchmark Team • Bravepoint • Dan Foreman • John Harlow • Progress • Gus Björklund

  6. Goals • Prove that Linux is stable and viable in the real world • Prove that price/performance is excellent • Produce configuration recommendations • And finally…...

  7. Why Linux ? • Commodity Priced Hardware • Inexpensive Operating System • Linux is no longer a toy • Currently being used in production (details to follow) in mission critical applications • Excellent alternative to Microsoft • Believers in Open Source • We don’t drink Microsoft Kool-Aid

  8. Why Not Linux ? • The constant temptation to apply the “patch of the day” • Credibility (the gap between perception of reality and reality) • Track record is relatively short

  9. Why Not Linux • It’s not like an OS where you call up your friendly 24 hr/day support line and they stick with fixing your problem; we posted a note to one of the Linux lists during our testing and never got a response; There are two ways to go: • the slow, plodding, conservative method • the complex, inter-dependent, patch-of-the-day method

  10. History • August 25, 1991: Linus Torvalds posts to comp.os.minix that he has his experimental kernel running gcc and bash. • “just a hobby, won’t be big and professional” • Version 0.01 sources posted September 1991 • 1.0 stable kernel release was March 1994 • SCO OpenServer binaries of Progress were being run on Linux as early as 1995 • Not supported by PSC • First Progress release for Linux (RedHat 6.2) was Version 8.3C in Spring 2000

  11. Example Production Linux Site • Transplatinum • Financial services provider for large trucking companies • Progress V9.1B • Linux 2.2.19-6.2.1 • DB Size 12GB • 137 Users • Only problem encountered: “getting the correct Linux patch combinations installed”

  12. Benchmark HardwareIn Zero Gravity

  13. Benchmark Server Hardware • Disks - 10 IBM Deskstar IDE (not SCSI) disk drives, 60 gigabytes each • 3Ware 7810 Disk Controller with H/W RAID • Memory - 2 gigabytes, PC133, SDRAM • CPUs - 2, 1 gigahertz Pentium III • ASUS Motherboard

  14. Hardware Cost (August 2001)

  15. Hardware Cost (1991)Sequent SMP Unix System

  16. Then Versus Now • Result Then: • 170 tps $4,925 per tps • Result Now: • 580 tps $7.60 per tps 3.4 x more performance1 / 648 the price per tps

  17. Software Details • Linux 2.4.18 Kernel • Stock except for the RAID array drivers • Progress V9.1C09

  18. Multi-million dollar benchmarking lab in a secret location

  19. Benchmark LaboratoryUnderground Bunker

  20. Benchmark Team (Partial)

  21. Benchmarks • Lies, True Lies, and Benchmarks • ATM benchmark • Didn’t compare performance to NT, W2K, or any other Intel Operating System

  22. ATM Benchmark • Simulates ATM withdrawal transaction • read, update account • read, update branch • read, update teller • create history • Execute as many times as possible in given time • Produces heavy update workload

  23. ATM Database • 4 tables • account: 20,000,000 records • branch: 20,000 records • teller: 2,000 records • history: create 1 per transaction • 3.2 gigabyte total size • 2 extents, 1,600 megabytes each • 2,300 megabytes initial data • 175 megabytes indexes

  24. Creating The 3.2 GB Database • prostrct create to allocate space • about 2 minutes • Creating the initial records via 4GL • 35 min • procopy to make backup • 10 minutes, including creating extents

  25. Other Times • Index rebuild (eventually) • 9 minutes, 10 seconds • proutil atm -C idxbuild all -TB 31 -TM 32 -B 250 -T /bi -t • sort file size: 381,086,720 bytes • dbanalysis • 6 minutes

  26. Documentation

  27. Baseline Run • Database size • 3 gigabytes • Everything on one disk • 8k database block size but otherwise untuned “out-of-the box installation”

  28. Baseline Run • Database size • 3 gigabytes • 4 tables • account, branch, teller, history • Everything on one disk • 8k database block size but otherwise untuned “out-of-the box installation” • Result: 30 tps (4 users)

  29. Top 8 Progress Tuning Items • 1. BI on a separate disk • 2. -spin 50,000 • 3. -B 32000 • Later we tried 64000 • 4. BI Cluster Size (-bi) 16 mb

  30. Top 8 Progress Tuning Items • 5. Page Writers (4) • 6. BI Writer • 7. BI Buffers (25+) • 8. 8K Database Block Size

  31. Top 8 Progress Tuning Items • 5. Page Writers (4) • 6. BI Writer • 7. BI Buffers (25+) • 8. 8K Database Block Size • Results before: 30 tps (4 users) • Results after: 61 tps (4 users)

  32. Linux Changes • Reiser FS • some of the best and worst numbers • too erratic so we didn’t continue • EXT2 versus EXT3 • no difference observed • noatime • no difference observed • Larger shared memory segments (32mb default raised to128 mb) • 2% gain

  33. Additional Progress Changes • -semsets • no statistically significant difference • Index rebuild of the test database • 3% gain

  34. Additional Progress Changes • Doubled -B from 32000 to 64000 (8k blocks) • Before: 557 tps (54 users) • After: 581 tps (58 users)

  35. Additional Progress Changes • File System Block Size/DB Block Size rated from best to worst: • 4k/8k • 4k/4k • 1k/8k - not tested • prostrct create took too long • 1k/1k- not tested • prostrct create took too long

  36. Disk Array Changes • JBOD • 8 disks • 479 tps (24 users) • RAID 0 • Striped 8 disks, 64k stripe size, bi on stripe set • 339 tps (26 users) • Moved BI file to internal (non-RAID) drives • 481 tps (32 users)

  37. Disk Array Changes • RAID 0 • 8 disks; just striping • -B 64000, index rebuild, raise shmmax • 581 tps (58 users) • Stripe & Mirror • 3 x 2 for data disks • 1 pair for BI • 304 tps (60 users)

  38. Disk Setups Compared

  39. Conventional Wisdom • There’s no such thing! • Direct I/O parameter reduced performance (551  118) but chopped off the spikes in the longest transaction duration; resorted to manual syncing similar to pre-directio days • Enabling On-board disk cache DECREASED Performance (506  474) • RAID 0 Stripe Size 1mb was much better than 64k

  40. Cost Of After Imaging • Before: 289 tps (92 users) • After: 278 tps (82 users) • 3.8% reduction

  41. Constraints • Time - infinite number of benchmarking possibilities • The unexpected: 3 hours to configure the RAID array from JBOD to RAID 10 • Ran out of beer on the last night • Occasional Disagreement between the Members of the Benchmarking Team • BI Buffers • What do we do next? • Where do we eat tonight?

  42. Linux Virtual Memory • bdflush • It’s in flux • Hurriedly overhauled in 2.4.10 • Will be overhauled again in 2.4.19, soon • Direct I/O does not produce results similar to other Unixen

  43. Miscellaneous Stuff • Can’t control the amount of OS Cache • at least, we could not

  44. What we Didn’t Do • RAID 5 - because as you know it’s Evil • Software striping & mirroring • Raw partitions • Non-standard file systems (JFS, etc) • 2.5 kernel • Timeslice parameters • NAS • Storage Areas • Records per Block

  45. ? Yes, but what about Windows

  46. Digression:Linux Vs. Windows Performance In a previous benchmark, we compared Linuxagainst Windows • Progress 9.1A05 • Windows 2000 Advanced Server @$3,500 USD • RedHat Linux 6.2, 2.2.14 kernel • Same exact hardware for both • IBM Netfinity 8500, 8 x 550 mhz Pentium III • 8 gigabytes RAM • 40 x 9 Gigabyte SCSI disks • 32 megabytes of disk cache

  47. Linux Vs Windows PerformanceResults (9.1A05) • Windows: 3,655 tps • Linux 4,508 tpsLinux was 23 % fasterLinux had more idle cpu ( ~ 15 %)

  48. Conclusions • Linux provides excellent performance • Could probably run 100 MFG/PRO users • System was easy to set up, easy to use • System cost was very low • We had no way to backup the system !! • 493 GB of disk in the array • Tape robot could cost $12,000 or more • Better than Windows on Intel

  49. What Should We Test Next Time ? • Next Linux Benchmark Oct 28, 2002 • Should we • compare Linux vs Windows with 9.1D? • compare AMD vs Intel ? • compare filesystems in detail? • try BIOS “tweaking”? • Ideas?

More Related