1 / 27

vgES Demonstrations

vgES Demonstrations. Andrew A. Chien, Henri Casanova , Fran Berman Yang-Suk Kee, Kenneth Yocum Richard Huang, Dionysis Logothetis, and Jerry Chou CSE, SDSC, and CNS University of California, San Diego VGrADS Site Visit April 28, 2005. vgES: Prototype Research Infrastructure.

frostd
Download Presentation

vgES Demonstrations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. vgES Demonstrations Andrew A. Chien, Henri Casanova, Fran Berman Yang-Suk Kee, Kenneth Yocum Richard Huang, Dionysis Logothetis, and Jerry Chou CSE, SDSC, and CNSUniversity of California, San Diego VGrADS Site Visit April 28, 2005

  2. vgES: Prototype Research Infrastructure • We have developed a functional prototype of vgES • vgES 0.7, March 2005 • Two demonstrations: • vgFAB: Finding and Selecting Resources • vgES: Full application run on the VGrADS testbed

  3. The vgES Research Prototype vgDL Application vgFAB

  4. The vgES Research Prototype vgDL Application vgFAB Resource Info vgAgent

  5. The vgES Research Prototype vgLAUNCH bind vgDL Application vgFAB Resource Info vgAgent

  6. The vgES Research Prototype vgLAUNCH bind vgDL Application vgFAB Resource Info vgAgent VG return VG to App

  7. Demonstration #1: vgFAB vgLAUNCH bind vgDL Application vgFAB Resource Info vgAgent VG return VG to App

  8. Demonstration #1: vgFAB vgLAUNCH bind vgDL Simple vgDL query program vgFAB Resource Info vgAgent VG return VG to App

  9. Demonstration #1: vgFAB vgLAUNCH bind vgDL Simple vgDL query program vgFAB Resource Info vgAgent VG return VG to App TeraGrid VGrADS

  10. VGrADS and TeraGrid Resources Xeon Itanium Rice UCSD UH UTK Itanium Pentium III VGrADS

  11. SDSC PSC CalTech ORNL VGrADS and TeraGrid Resources Itanium Itanium Itanium ANL Itanium Purdue Power4 IU Xeon Power3 Xeon Itanium Altix Xeon Itanium Pentium 4 Rice Alpha UCSD NCSA TACC TeraGrid UH UTK UltraSparc Itanium Pentium III VGrADS Itanium

  12. SDSC PSC CalTech ORNL A simple vgDL query Itanium Itanium Itanium VG1 = ClusterOf(node) [4:64] { node =[ (Processor == Xeon) && (Clock >= 1000) && (Memory >= 1000) ] } ANL Itanium Purdue Power4 IU Xeon Power3 Xeon Itanium Altix Xeon Itanium Pentium 4 Rice Alpha UCSD NCSA TACC TeraGrid UH UTK UltraSparc Itanium Pentium III VGrADS Itanium

  13. Switch to live demo

  14. SDSC PSC CalTech A simple vgDL query Itanium Itanium Itanium VG1 = ClusterOf(node) [4:64] { node =[ (Processor == Xeon) && (Clock >= 1000) && (Memory >= 1000) ] } ANL Itanium Purdue Power4 IU Xeon Power3 Xeon Itanium ORNL Altix Xeon Itanium Pentium 4 Rice Alpha UCSD NCSA TACC TeraGrid UH UTK UltraSparc Itanium Pentium III VGrADS Itanium

  15. SDSC PSC CalTech ORNL A more complex vgDL query Itanium Itanium Itanium VG2 = rsc1 = ClusterOf(node)[4:64] { node = [ (Processor == Xeon) && (Clock>=1000) && (Memory >=1000) ] } FAR rsc2 = LooseBagOf(cluster1)[1:20] { cluster1 = ClusterOf(node)[4:128] { node = [ (Processor == Itanium) && (Memory >= 2048) ] } } ANL Itanium Purdue Power4 IU Xeon Power3 Xeon Itanium Altix Xeon Itanium Pentium 4 Rice Alpha UCSD NCSA TACC TeraGrid UH UTK UltraSparc Itanium Pentium III VGrADS Itanium

  16. Switch to live demo

  17. PSC A more complex vgDL query Itanium Itanium Itanium VG2 = rsc1 = ClusterOf(node)[4:64] { node = [ (Processor == Xeon) && (Clock>=1000) && (Memory >=1000) ] } FAR rsc2 = LooseBagOf(cluster1)[1:20] { cluster1 = ClusterOf(node)[4:128] { node = [ (Processor == Itanium) && (Memory >= 2048) ] } } VG1 = foo + foo + ANL SDSC Itanium Purdue Power4 IU Xeon Power3 SHOW RESULTS Xeon Itanium CalTech ORNL Altix Xeon Itanium Pentium 4 Rice Alpha UCSD NCSA TACC TeraGrid UH UTK UltraSparc Itanium Pentium III VGrADS Itanium

  18. Synthetic Resource Environments • Resource selection is a difficult problem • It’s NP-hard • We have a heuristic and a prototype implementation • Research: result quality, scalability, response time, contention, ... • What is needed: Simulation studies in large and realistic environments • We have developed a “resource environment generator” • Based on survey of existing systems and analysis of technology trends • Kee, Casanova,Chien, Realistic Modeling and Synthesis of Resources for Computational Grids, SC2004. • A sample synthetic environment • 1 million hosts, 10,000 clusters, Pentium 2-4, Itanium, Opteron, Athlon. • VG3 = rsc1= ClusterOf (node) [4:64] { node = [ (Processor==Pentium4) && (Clock>=2000) && (Memory>=4096) ] } FAR rsc2 = LooseBagOf (nest_cluster) [1:20] { nest_cluster = ClusterOf (node) [4:128] { node = [ (Processor==Itanium) && (Memory>=8192) ] } }

  19. Switch to live demo

  20. Take-away from Demonstration #1 • We have a working research prototype for vgFAB within vgES • interfaces with the application via vgDL • interfaces with resource information systems • returns sets of matching resources • Makes it possible to use a high-level description of resource requirements • Makes it possible to find resources over different resource environments • Research • evaluating scalability, result quality, etc. [CCGrid’05] [SC’05 submission]

  21. Demonstration #2: vgES vgLAUNCH bind vgDL Application vgFAB Resource Info vgAgent VG return VG to App

  22. Demonstration #2: vgES vgLAUNCH bind vgDL Application vgFAB Resource Info vgAgent VG return VG to App

  23. Application: EMAN • EMAN: Electro Micrograph ANalysis • Performs 3-D reconstructions • Workflow Application • See Chuck Koelbel’s talk this afternoon, and the EMAN poster • Demonstration: Run EMAN on the VGrADS testbed with vgES Xeon Itanium seq Rice UCSD par UH UTK Itanium Pentium III

  24. EMAN and the Virtual Grid VGES myVGES = new VGES() // new vgES instance myVG = myVGES.createVG( vgDL_string ) // found & bound VG vgRoot = myVG.getRoot(); ... // Traverse VG tree to find resource information vgES.copyToNode(someNode, “input1”); // Send input files vgES.runCmd(someNode,”command”); // Start command vgES.copyFromNode(someNode, “output1”); // Get output files vgES.terminateVG(myVG); // Destroy VG VG = LooseBagOf (cluster) [1:4] { cluster = ClusterOf (node) [4:10] { node = [Clock >= 900] } }

  25. Switch to live demo

  26. Take-away from Demonstration #2 2-D images 3-D model • The vgES prototype is functional for a real-application • VG provides a simple abstraction that integrates • access to resources (Globus) • access to resource information (MDS, Ganglia, NWS, etc.)

  27. VGrADS and the Virtual Grid • The VG abstraction and its runtime implementation are the focal point of the VGrADS multi-team collaboration • see afternoon presentations and posters • Interface to applications (vgDL + VG) • Program preparation and optimization • Application scheduling • Monitoring, fault-tolerance, and adaptation • Resource Management • Single access/interface to various resource information sources • Research platform for all the above VG

More Related