1 / 10

Amen Anon et. al.

Amen Anon et. al. HPTS 99 Are TP Benchmarks Still Relevant to the TP Community?. Summary. Benchmarks drove much TP technology TPC benchmarks are not relevant to the rapid growth and rapid change of the Web environment Benchmarks are not challenging to the TP technical community.

Mia_John
Download Presentation

Amen Anon et. al.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Amen Anon et. al. HPTS 99 Are TP Benchmarks Still Relevant to the TP Community? Bill Laing

  2. Summary • Benchmarks drove much TP technology • TPC benchmarks are not relevant to the rapid growth and rapid change of the Web environment • Benchmarks are not challenging to the TP technical community Bill Laing

  3. TPC Galvanized the TP Community • Historically many of the base TP technology issues were developed to address high TP rates and win benchmarks • Locking strategies • Group commit • Log striping • Undo/redo strategies • Etc. etc. • See previous HPTS papers • Along time ago 1000 (debit/credit) TPS was a dream and a motivator • Now it is a common place vendor capability • So what now? • Have we lost our way? Bill Laing

  4. TPC-C TPC-C is still the “fairest” benchmark for competition but it is • Losing relevance • Unrealistic configurations (RAID0 only, No failover) • Mostly a database focus • Measures price/performance and absolute performance but has no measure of: • Application Development costs • Deployment costs • Operations costs • Application Maintenance • Availability • Cost and ease of scalability Yes there are H (1 result), R (1 result), and W is coming but • Are they relevant to TP technologists? Bill Laing

  5. TPC-C results Top single system #164 CPUs 115396 (Sun/Oracle) #4 32 CPUs 102023 (HP/Sybase) #5 64 CPUs 93901 (Sqnt/Oracle/Dynix) #6 32 CPUs 92833 (HP/Oracle/HP-UX) #7 24 CPUs 53050 (Sun/Sybase) #8 16 CPUs 52117 (HP/Sybase/HP-UX) Scale out # 2 5 x 12 CPUs 110434 (IBM/Oracle/AIX) # 3 8 x 12 CPUs 102542 (Alpha/Oracle/Unix) # 9 2 x 22 CPUs 51871 (Sun/Oracle/Solaris) #10 4 x 8 CPUs 50208 (NT/NEC/Oracle) #21 4 x 4 CPUs 33936 (NT/Compaq/Oracle) #24 6 x 4 CPUs 27383 (NT/Tandem ServerNet/Oracle) #79 4 x 8 CPUs 14286 (IBM/AIX/PPC) Bill Laing

  6. 27 Sun StorageEdge A3500 with 1778 disks (15.6 TB) Sun E10000 + Solaris+ Oracle8i 64x400 Mhz cpus + 64 GB RAM (8,430k$ hardware + 1,890 k$ software) Tuxedo® on 32 Sun Ultra 10 333Mhz workstations, Wyse terminals 429k$ hardware, 168k$ software The Biggest TPC-C BenchmarkSun+Oracle115,395.73 tpmC @ 105.63 $/tpmC == 12.2 M$ available 8/22/99 Bill Laing

  7. TPM-C Relevance? How much is 100 kTpmC? • 100,000 users • Each submitting 2.3 transactions/minute • About 300 M transactions/day • Greater than Yahoo’s page views during Christmas last year • And remember • Over 90% of the TPC-C transactions are buy vs. browse • Average web site is less than 10% buy vs. browse Bill Laing

  8. What happened to Scale Out • Most results are not scale out • Most Web sites start small and want to grow dramatically • So far scale out hasn’t won • Is it what customers want? Bill Laing

  9. New Benchmark • Food for discussion during the next 2 days • Benchmark • Should focus on driving TP technology • Should show incremental growth • Closest to linear growth? Bill Laing

  10. Summary • Benchmarks drove much TP technology • TPC benchmarks are not relevant to the rapid growth and rapid change of the Web environment • Benchmarks are not challenging to the TP technical community Bill Laing

More Related