1 / 29

The MicroGrid: A Scientific Tool for Modeling Grids

The MicroGrid: A Scientific Tool for Modeling Grids. Andrew A. Chien SAIC Chair Professor Department of Computer Science and Engineering University of California, San Diego April 30, 2001. Outline. Motivation What is a MicroGrid? Validating Models Status Future Work. Motivation.

hong
Download Presentation

The MicroGrid: A Scientific Tool for Modeling Grids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The MicroGrid: A Scientific Tool for Modeling Grids Andrew A. Chien SAIC Chair Professor Department of Computer Science and Engineering University of California, San Diego April 30, 2001

  2. Outline • Motivation • What is a MicroGrid? • Validating Models • Status • Future Work

  3. Motivation • Need tools to study complex dynamic Grid behavior • complex non-linear dynamic behavior • Tightly couple communication, computing, and storage resources • Performance, Availability, Failure • Complementary approaches useful, but insufficient • MacroGrids • Limitations of scale and actual configuration • Major logistical efforts • Other Simulations • Network-only (internet/networking) • Application level (simple resource models) • Enable design of robust, reliable, good performing Grids and Grid applications

  4. Grid Application Developer “Cactus” • How will my software behave on the projected hardware configuration? (performance) • How will it behave dynamically? (robustness) • How will it interact with other Grid applications an uses of the system? • How can I make this a robust, stable, reusable application? “Zeus-MP” “Tardis” “Netsolve” “GTomo” “SF-Express” “Distributed Viz”

  5. Grid System Software Developer • Libraries – network, performance instrumentation, runtime environment (e.g. Globus) • Program Preparation System – dynamic compilers, runtime, etc. • Do these things work and how well? • With what applications and what range of applications? “GrADS” “NWS” “PPS” “Globus” “Nimrod” Grid Researchers

  6. Grid System Administrator • What if I change my resource access policies? • What if I add/take away these resources? • What if I change the “price” charged for resources? • What happened to my Grid when it melted down last week?

  7. MicroGrid Goals • Runtime environment for GrADS experiments (a la MacroGrid) • Develop technology and tools to support specialized Grid communities (a la MacroGrid) • Realistic modelling of a broad range of Grid systems, applications, environments, and dynamic behavior • Execution of real applications (tools and middleware) • Scale to large experiments • High fidelity simulation, support variety of speed + fidelity • Network, compute, memory, disk • Observable, repeatable behavior

  8. Outline • Motivation • What is a MicroGrid? • Validating Models • Status • Future Work

  9. MicroGrid Modeling Grid Application • A scientific tool for modeling Computational Grids • Run arbitrary Grid applications on any virtual Grid resources • Allow the study of complex dynamic behavior of large systems Virtual Grid MicroGrid Software LAN Workgroup Scalable Cluster Heterogeneous Environment

  10. MicroGrid Today • Processor speed modeling • Memory size modeling • Virtualized Resource description (GIS/MDS) • Network Virtualization • Online Network Simulation • => runs the Globus 1.1.3 software • => runs Globus applications on a Linux/Alpha testbed

  11. Grid Application Virtual Grid, “MicroGrid” MicroGrid Software Using a MicroGrid • Find some physical resources • Configure a Virtual Grid • Submit a Globus Job to it • Observe Execution (which occurs in virtual time) • DeConfigure the Virtual Grid

  12. Outline • Motivation • What is a MicroGrid? • Validating Models • Status • Future Work

  13. MicroGrid Validation • Simulate an benchmarks and applications • various Grid systems • Run simulations on the physical hardware • Compare to published results

  14. Validation on Micro-benchmarks • Memory Capacity Modeling • Processor Speed Modeling • NSE Network Modeling • Each resource model is validated

  15. Validation on NPB Benchmarks • Comparison to published cluster NPB results • Set parameters based on known published relative resource performance -- processor and network performance • Alpha cluster (Alpha’s + 100Mbit Ethernet) and HPVM cluster • Overall execution time matches within 4%

  16. NPB over WAN • vBNS • A fictional Cluster • Varying WAN bandwidth

  17. NPB over WAN (Cont.) • No background network traffic • Performance is insensitive to network bandwidth • Shows a simulation of hypothetical cluster on WAN

  18. Internal Behavior of NPB • Autopilot tools for Program Tracing (in MicroGrid environment) • Traces from MicroGrid and real Grid • Match within 5%

  19. Validation on Large Applications • Cactus PDE Solver Framework on Alpha cluster • WaveToy program, various Matrix sizes • Execution time matches within 7%

  20. Outline • Motivation • What is a MicroGrid? • Validating Models • Status • Future Work

  21. MicroGrid Today • Uses Globus 1.1.3 • Supports Globus 1.1.3 applications and tools • Incorporates models for • Processor speed • Memory capacity • Virtualized Resource Description (GIS/MDS) • Network Virtualization • Online Network Simulation • Used via standard submission interfaces • Not yet available for external users, improving robustness and adding modules

  22. What have we learned? • Demonstrated accurate simulation of Grid environments and applications • Demonstrated ability to support existing applications and tools (critical for significant experiments) • Existing network simulation tools are inadequate • Existing network traffic models are inadequate • Deriving network configuration information is challenging • Extrapolation of results is a major challenge due to nonlinearity of behavior

  23. What have we learned? (cont) • There’s a LOT more work to be done to support • large-scale, high speed simulations, • with flexible choice of resource models, • simulating a wide range of environments (config, background activity, etc.), and • executing on a wide range of physical hardware resources.

  24. Milestones Year 1: • Develop Initial Version of MicroGrid toolkit • Empirical study of application behavior based on MicroGrid toolkit Year 2: • GrADS runtime environment and applications on the MicroGrid (in progress)

  25. Outline • Motivation • What is a MicroGrid? • Validating Models • Status • Future Work

  26. Ongoing and Future Activities • System Development (Better MicroGrid) • Scalable On-line Network simulation – Xin “Paff” Liu • Variable speed simulation (efficiency) – Ranjita Bhagwan • Network Traffic Modeling (background & coupled load) – Xianan Zhang • Disk Speed Modeling (I/O intensive workloads) – Huaxia Xia • Other current activities (Validation, Software) • Scalapack modeling – Match GrADS results • Cactus modeling – Match GrADS results • Porting to x86 Linux • Robustify and package for external release

  27. Summary • Demonstrated that MicroGrid approach can produce accurate results in modeling • Grid applications • Grid infrastructures • Dynamic behavior • Working software • Significant validation • Micro-benchmarks; Full benchmarks; Applications • … Need to get MicroGrid software to the next level of capability …

  28. MicroGrid Team • Dr. Andrew Chien (PI) • Graduate Students: • Xin “Paff” Liu, Ranjita Bhagwan, Xianan Zhang, Huaxia Xia • Former: • Dr. Hyo Jung Song (Postdoc) • Dr. Kenjiro Taura (U Tokyo Professor) • Dennis Jakobsen (MS) • For more information see • http://hipersoft.rice.edu/grads/project/micro.html • http://www-csag.ucsd.edu/

More Related