1 / 17

Parallel File System Benchmarking Project

Parallel File System Benchmarking Project. Tracey Wilson DICE, Program Manager CSC twilso23@csc.com. Outline. DICE Overview What is the DICE TAP? The File System Comparison Issue and Why? DICE Plan for Framework Normalization Summary.

amy
Download Presentation

Parallel File System Benchmarking Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel File SystemBenchmarking Project Tracey Wilson DICE, Program Manager CSC twilso23@csc.com

  2. Outline • DICE Overview • What is the DICE TAP? • The File System Comparison Issue and Why? • DICE Plan for Framework • Normalization • Summary

  3. Nationwide Test Environment for Technology Innovations

  4. Testing & Validation ServicesIndependent 3rd Party Evaluations Evaluating HPC, Storage, Networking and Data Management Technologies

  5. DICE Architecture • DICE utilizes a geographically distributed test bed • Real world research networks • DICE sites indicative of HPC Data Centers

  6. Quite Simply…A Test Bed for Advancing Innovation Independent, Unbiased 3rd Party Evaluations Genesis of a Vision by Government, Industry & Academia to Address Critical HPC Technology Challenges

  7. Community Input is Key • The DICE Technical Advisory Panel • Advises DICE on: • Technology directions • Project concepts • Testing methodologies • Utilizes working groups to focus on more specific issues • Current members include:

  8. What is the File System Issue? • Lack of standardized metrics for performance • Need exists for file system performance comparison • Need exists for defined benchmarks for local and remote file systems • Different HPC systems and their file system architectures have no direct correlations • Scaling: • Scaling of current file systems has unpredictable performance impacts • Load: • Performance varies with file systems with loads of 50-70% • Fragmented data is not accounted for

  9. Why do HPC Centers Care? • Centers need input for: • New procurements • User requirements • Application performance • File systems are a critical piece of an HPC solution • How do you as a Center Director make an informed decision? “Can you prove or rather disprove the performance or value of one file system compared to another?”

  10. How did the Effort Begin? • In 2007, DICE received 3 proposals for evaluation projects: • Panasas • DoD • Microsoft • DICE Advisory Panel offered new solution • “Develop a standard benchmark framework for file system evaluation” Make the framework expandable to use Make it fully available for the HPC community to use Effort was to include normalization of results

  11. Benefits of Standardization • Standardized Metric • One set of benchmarks for all parallel file systems • Results will be normalized for comparison • HPC Community needs non-biased benchmark(s) • Increased understanding of the impacts on current and proposed upgrades to a center’s file system: • The storage subsystem performance • Scaling • Load performance

  12. Current Ideas • Parameterized Benchmark Suite • Simulate many different workloads • Expose more tuning/configuration options in the benchmarks • Suite Extensible so users can add new benchmarks • Have unique benchmarks for synthetic workloads • Transactions • Streaming • Random I/O • Various Read/Write Ratios

  13. What is the plan? • Currently surveying the HPC benchmark and file system community including: • Carnegie Melon • University of California/Santa Cruz • ORNL • Argonne National Labs • PNNL • TACC • LLNL • And many more

  14. Application I/O Patterns • Many different I/O intensive applications • Investigate the development of an I/O analyzer • Recognize I/O utilization of applications • Similar to trace and trace replay • Add this functionality to the framework to simulate application I/O patterns • Verification of performance • Can be applied on different architectures

  15. Normalization • Normalization is needed between differing architectures • Need a formula • Performance • Cost • Additional Criteria will be evaluated • Aging and fragmentation of data should be considered • Difficult to develop • Need to develop on a small scale first • Will the normalization hold as you scale systems? • Plan is to create the framework first then normalize

  16. Summary • Lot of collaborative effort from the Team and the Community is required • Full project plan is in draft • Survey underway to identify current efforts • Plan to have initial framework for file system testing by early 2010 • Plan to use this on other file system types

  17. How To Get Involved • Collaboration is key • Developing a DICE Forum for this project • Will be collecting traces • DICE TAP will be forming specific working group for this effort • Interested parties should send email to twilson@avetec.org • DICE Website www.diceprogram.org

More Related