1 / 22

AppLeS, NWS and the IPG

This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation In Slide Show, click on the right mouse button Select “Meeting Minder” Select the “Action Items” tab

wynona
Download Presentation

AppLeS, NWS and the IPG

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation • In Slide Show, click on the right mouse button • Select “Meeting Minder” • Select the “Action Items” tab • Type in action items as they come up • Click OK to dismiss this box This will automatically create an Action Item slide at the end of your presentation with your points entered. AppLeS, NWS and the IPG Fran Berman UCSD and NPACI Rich Wolski UCSD, U. Tenn. and NPACI

  2. AppLeS and the IPG Usability, Integration development of basic IPG infrastructure Development of persistent IPG testbed Experience with Pilot IPG Development of prototype performance-oriented applications Development of necessary research Performance “IPG - aware” programming Short-term Medium-term Long-term Integration of schedulers and other tools, performance interfaces Application scheduling Resource scheduling Throughput scheduling Multi-scheduling Resource economy

  3. Performance feedback Perf problem Realtime perf monitor Software components Service negotiator Grid runtime system Config. object program Source appli- cation whole program compiler P S E negotiation Scheduler Dynamic optimizer libraries A Model for the Future • Adaptation is key to the ultimate IPG program development and execution environment. • Exchange of performance information fundamental to the success of IPG applications Grid Application Development System (GrADS)

  4. Why Application Schedulers? • Application performance can conflict with performance goals of other system components • Goal of application scheduler is to prioritize performance of the application over other system components

  5. NWS User Prefs App Perf Model Sensor Interface Resource Selector Planner Reporting Interface Application Forecaster IPG /Globus Act. infrastructure Model Model Model Agent-based Application Scheduling AppLeS(Berman and Wolski) NWS(Wolski)

  6. Performance Prediction • Given monitored bandwidth data, what will happen next?

  7. NWS Predictions • Monitored data provides a snapshot of what has happened. • What we really want to know is: What will happen?

  8. Monitoring vs. Prediction • Last value not always the best predictor • Hard to develop accurate forecasting models -- why not use all feasible models? Monitored data

  9. Do AppLeS and NWS Improve Application Performance? • Good results with many applications including • SARA AppLeS • CompLib AppLeS • Jacobi2D AppLeS • AppLeS/NWS applications demonstrate that • prediction is possible in high-variance environments • adaptivity can improve performance

  10. SARA AppLeS • SARA = Synthetic Apperture Radar Atlas • application developed at JPL and SDSC • Goal: Process radar images from distributed database for user’s desired image • AppLeS focuses onresource selection problem

  11. SARA Experiments Data Servers Compute Servers Client . . .

  12. sequence library sequence library CompLib AppLeS • Problem: Find the best matches between two gene sequence libraries • Apply FASTA algorithm to all sequence pairs to determine similarity • Developed for DOCT testbed

  13. CompLib Experiments

  14. Jacobi2D AppLeS • Important component of many scientific applications • Time-balancing used to achieve minimal execution time • Scheduler solves time-balancing equations for Area

  15. Jacobi2D Experiments • Comparison of AppLeS with and without NWS info, and load-balancing

  16. Applying AppLeS/NWS Methodology to the IPG • AppLeS/NWS methodology can be used to develop performance-efficient IPG applications • IPG FY99 projects leverage FY98 project and previous AppLeS/NWS development and research

  17. Act Exp App-specificcase gen. API Sched. Exp Exp Act S AppLe Act Resources IPG FY99 Project: A “Parameter Sweep” Template • INS2D representative of larger class of critical NASA applications • AppLeS parameter sweep template will build on INS2D model and experiments to target larger class of applications and platforms • Template will serve as a prototype IPG PSE workbench tool

  18. AppLeS Project Plan FY99 (Berman,UCSD) • Expand INS2DAppLeS • to NASA IPG testbed • to include batch systems • to target Globus • Development of Parameter Sweep AppLeS template • Goal: To provide framework for improving turnaround time of parameter study component of complex AES applications • AppLeS scheduling agents prototype autonomous agent technology for IPG • Requires development of strategy for scheduling in mixed batch and interactive environments Project Personnel: Berman, Casanova (UCSD) Collaborators: Wolski (U. Tenn.), Kesselman (ISI/USC)

  19. NWS Project Plan FY99(Wolski, U. Tenn.) • Enhance the NWS to support AppLeS parameter sweep template in NASA Globus environment • NWS API for parameter sweep template • integration with Globus • Integrate NWS with IPG and Globus application performance monitoring tools • use NWS performance techniques to predict application performance dynamically • Investigate strategies for monitoring and forecasting batch system performance • queue wait times in the presence of user priorities, etc. Project Personnel: Wolski (U. Tenn) Collaborators: Berman (UCSD), Moore (SDSC), Kesselman (ISI/USC)

  20. Possible Additional IPG Projects • AppLeS/NWS-enhanced Storage Resource Broker Project:Enhance SRB performance through agent-based scheduling Project Personnel: Berman, Wolski Collaborator: Moore • AppLeS/NWS-enhanced NetSolve over Globus Project: Improve scheduling component of NetSolve using AppLeS/NWS techniques, deploy on Globus IPG platform Project Personnel: Berman, Wolski, Casanova, Dongarra Collaborator: Kesselman

  21. Possible Additional IPG Projects • AppLeS/NWS Applications on Condor Project:Develop AppLeS application which can achieve performance in the Condor environment; integrate Condor and NWS information; leverage Condor/Globus integration Project Personnel: Berman, Wolski Collaborator: Livny, Kesselman

  22. NWS Home Page: http://nws.npaci.edu AppLeS + NWS Project Personnel Francine Berman Rich Wolski Walfredo Cirne Marcio Faerman Jaime Frey Jim Hayes Graziano Obertelli AppLeS Home Page:http://www-cse.ucsd.edu/groups/hpcl/apples.html Jenny Schopf Gary Shao Neil Spring Shava Smallen Alan Su Dmitrii Zagorodnov Project Information

More Related