1 / 46

Virtual Private Grid: A Command Shell for Utilizing Hundreds of Machines Efficiently

Virtual Private Grid: A Command Shell for Utilizing Hundreds of Machines Efficiently. Kenji Kaneda Kenjiro Taura Akinori Yonezawa University of Tokyo. Outline of Presentation. Introduction Virtual Private Grid (VPG) Implementation of VPG Experiments Summary Demonstration.

nevan
Download Presentation

Virtual Private Grid: A Command Shell for Utilizing Hundreds of Machines Efficiently

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virtual Private Grid: A Command Shell for Utilizing Hundreds of Machines Efficiently Kenji Kaneda Kenjiro Taura Akinori Yonezawa University of Tokyo

  2. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  3. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  4. Lab. Univ. My Home Background • People have many computational resources • Super Computers • Clusters of Workstation/PC • etc. • People want to run parallel programs • e.g.) parameter-sweep-application

  5. Ancient Internet • No administrative restrictions • e.g.) No firewall, DHCP client, private IP, … • Each machine had a unique IP address • Machines were able to communicate directly with each other • Job submission was very easy > rsh 133.11.12.200 ps 5762 pts/10 0:00 tcsh > rsh 133.11.12.201 ps 5763 pts/10 0:00 tcsh > rsh 133.11.12.202 ps 5764 pts/10 0:00 tcsh 133.11.12.200 133.11.12.201 133.11.12.202

  6. Today's Internet • Various administrative restrictions • e.g.) firewall, DHCP client, private IP • Machines have no unique name • Machines cannot communicate directly with each other • Job submission becomes cumbersome private IP DHCP client Firewall

  7. Examples of Administrative Restrictions (1/3) • Firewall • It restricts access from external hosts > rsh host1 ps Error: cannot connect to host0 > rsh gateway0 > ssh gateway1 > rsh host1 ps 5764 pts/10 0:00 tcsh host0 host1 Firewall gateway0 gateway1

  8. Examples of Administrative Restrictions (2/3) • DHCP client • Its IP address changes dynamically > rsh 133.11.12.200 ps DHCP client 5764 pts/10 0:00 tcsh > rsh 133.11.12.200 ps Error: cannot connect to 133.11.12.200 > rsh 133.11.12.199 ps 5764 pts/10 0:00 tcsh 133.11.12.199 133.11.12.200

  9. Examples of Administrative Restrictions (3/3) > rsh 192.168.0.1 ps • Firewall + Private IP • It restricts access from external hosts Error: cannot connect to 192.168.0.1 > rsh gateway0 > ssh gateway1 > rsh gateway2 > rsh 192.168.0.1 ps 5764 pts/10 0:00 tcsh private IP Firewall 192.168.0.1 gateway0 gateway1 gateway2

  10. Utilizing Remote Machines is Difficult • User need to work around administrative restrictions • e.g.) connecting all the machines by keeping TCP connections permanently • Gateways keep a connection between them • DHCP client initiates a connection to outside • Private host initiates a connection to outside • It is impossible to handle all the restrictions manually • as the number of machines increases • e.g.) It is difficult to select only necessary connections to create a single connected graph

  11. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  12. Virtual Private Grid (VPG) • Command shell • User can access remote machines directly and transparently • e.g.) command@host • VPG automatically constructs a self-stabilizing spanning tree • VPG requires only a small number of connections • New tree is constructed whenever network topology changes

  13. host4 host6 host0 host3 host5 host1 host2 Features (1/4) • Nicknaming • VPG gives each machine a unique name • Independent from a DNS name and IP address

  14. Standard input is forwarded Output is forwarded Features (2/4) • Remote job submission • e.g.) ps@host4 'ps' is executed host4 host6 host0 host3 host5 host1 host2

  15. Features (3/4) • Redirection from/to file on remote machine • e.g.) cat@host4 < fileA@host0 > fileA@host6 • Pipe between remote machines • e.g.) tar@host2 –c file | tar@host5 x • Shell script • e.g.) parameter-sweep-application host4 host6 host1 host3 host5 host0 host2

  16. Features (4/4) • No modification of administrative policy • VPG tolerates • dynamic addition/removal of machines • dynamic disconnection between machines host4 host6 host1 host3 host5 host0 host2

  17. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  18. Overview of Implementation • Daemons start on all the available machines • creating and keeping some TCP connections • constructing a tree • Shell starts on a user's local host • keeping track of network topology • calculating a shortest-path to the target Shell

  19. Other Implementation Issues (1/2) • Daemon Configuration • User must give following configuration information manually • Nickname • Port number • List of possible connections

  20. Other Implementation Issues (2/2) • Security Issues • Many firewalls allow only SSH to login • VPG uses SSH connection if necessary • Public key authentication with empty pass-phrase • We are planning to authenticate connections between daemons • restricting access from malicious users

  21. Tree Construction Algorithm (1/3) • We uses self-stabilizing spanning tree algorithm [S. Kutten et al.] • Each node knows only its possible connections • Each node initiates some connections to its neighbors • Nodes gradually coalesce into large trees • Eventually, all the nodes construct a single spanning tree • Nodes construct a new tree whenever network topology changes dynamically possible connection kept connection

  22. Tree Construction Algorithm (1/3) • We uses self-stabilizing spanning tree algorithm [S. Kutten et al.] • Each node knows only its possible connections • Each node initiates some connections to its neighbors • Nodes gradually coalesce into large trees • Eventually, all the nodes construct a single spanning tree • Nodes construct a new tree whenever network topology changes dynamically possible connection kept connection

  23. Tree Construction Algorithm (1/3) • We uses self-stabilizing spanning tree algorithm [S. Kutten et al.] • Each node knows only its possible connections • Each node initiates some connections to its neighbors • Nodes gradually coalesce into large trees • Eventually, all the nodes construct a single spanning tree • Nodes construct a new tree whenever network topology changes dynamically possible connection kept connection

  24. Tree Construction Algorithm (1/3) • We uses self-stabilizing spanning tree algorithm [S. Kutten et al.] • Each node knows only its possible connections • Each node initiates some connections to its neighbors • Nodes gradually coalesce into large trees • Eventually, all the nodes construct a single spanning tree • Nodes construct a new tree whenever network topology changes dynamically possible connection kept connection

  25. Tree Construction Algorithm (2/3) • How to merge trees • Each tree has a unique priority • Tree with higher priority overruns a tree with lower priority • Eventually, single tree with the highest priority is constructed 0 17 11 12 2 7 13 6 1 3 16 14 10 9 5 15 4 8

  26. Tree Construction Algorithm (2/3) • How to merge trees • Each tree has a unique priority • Tree with higher priority overruns a tree with lower priority • Eventually, single tree with the highest priority is constructed 17 10 13 6 3 15 16 9 5 4 8

  27. Tree Construction Algorithm (2/3) • How to merge trees • Each tree has a unique priority • Tree with higher priority overruns a tree with lower priority • Eventually, single tree with the highest priority is constructed 17 10 16 15

  28. Tree Construction Algorithm (2/3) • How to merge trees • Each tree has a unique priority • Tree with higher priority overruns a tree with lower priority • Eventually, single tree with the highest priority is constructed 17

  29. Tree Construction Algorithm (3/3) • Possible connections need not be very precise • Even if possible connections are not listed, VPG works correctly • Even if blocked connections are listed, VPG works correctly • This information is currently required only for performance • We are planning to detect it dynamically

  30. Tree Construction Algorithm (3/3) • Possible connections need not be very precise • Even if possible connections are not listed, VPG works correctly • Even if blocked connections are listed, VPG works correctly • This information is currently required only for performance • We are planning to detect it dynamically

  31. Tree Construction Algorithm (3/3) • Possible connections need not be very precise • Even if possible connections are not listed, VPG works correctly • Even if blocked connections are listed, VPG works correctly • This information is currently required only for performance • We are planning to detect it dynamically

  32. Tree Construction Algorithm (3/3) • Possible connections need not be very precise • Even if possible connections are not listed, VPG works correctly • Even if blocked connections are listed, VPG works correctly • This information is currently required only for performance • We are planning to detect it dynamically

  33. Tree Construction Algorithm (3/3) • Possible connections need not be very precise • Even if possible connections are not listed, VPG works correctly • Even if blocked connections are listed, VPG works correctly • This information is currently required only for performance • We are planning to detect it dynamically

  34. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  35. Experimental Environment • Network environment • 3 subnets • Various architectures • Sparc, x86, MIPS, PowerPC • Various operating systems • Solaris, Linux, IRIX • VPG ran on approximately 100 nodes

  36. Comparison to Other Job Submission Tools (1/2) • Job submission tools • globus-job-run [I. Foster et.al.] • SSH • rsh • VPG • Measuring overhead of job submission • Turn around time of a light-weight job submission • Multi-hops job submission • e.g.) ssh hostA 'ssh hostB command'

  37. Comparison to Other Job Submission Tools (2/2) • globus-job-run, SSH, and rsh require authentication, whenever submitting a job • Authentication of globus-job-run and SSH causes a large overhead • VPG does not require authentication, whenever submitting a job • VPG requires authentication only once when creating a tree

  38. Low Level Interface • We derive a communication library out of VPG implementation • vpg_connect() • vpg_accept() • vpg_send() • vpg_recv() • We ran ray-tracing program on 32 nodes

  39. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  40. Related Work (1/2) • Globus [I. Foster et.al.] • No build-in function for working around administrative restrictions • Condor [M. Livny et.al.] • No build-in function for working around administrative restrictions • SSH [http://www.ssh.com] • Difficult to utilize hundreds of machines

  41. Related Work (2/2) • SOCKS [www.socks.nec.com] • General framework to bypass a firewall • No naming scheme for private IP nor DHCP client • Difficult to manage machines over many subnets • Virtual Private Network (VPN) • Mechanism to connect multiple subnets via Internet • It requires modification of administrative policy

  42. Summary • Virtual Private Grid • utilizing many machines over multiple subnets • by working around administrative restrictions automatically • available at • http://web.yl.is.s.u-tokyo.ac.jp/~kaneda/vpg

  43. Future Work • Simplification of daemon configuration • e.g.) removal of a list of possible connections • by modifying tree-construction algorithm • Automatic resource selection • Simple task mapping algorithm • Location of input/output file • Communication through pipes

  44. Outline of Presentation • Introduction • Virtual Private Grid (VPG) • Implementation of VPG • Experiments • Summary • Demonstration

  45. Demonstration • Job submission through VPG • Dynamic tree construction

More Related