1 / 35

A Short Introduction to PVM and MPI

A Short Introduction to PVM and MPI. Philip Papadopoulos University of California, San Diego Department of CSE San Diego Supercomputer Center. Outline. What is message passing? Why do I care? “Hello-World” for message passing Level 0 Issues What are PVM and MPI? MPI Implementations

koren
Download Presentation

A Short Introduction to PVM and MPI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Short Introduction to PVM and MPI Philip Papadopoulos University of California, San Diego Department of CSE San Diego Supercomputer Center

  2. Outline • What is message passing? Why do I care? • “Hello-World” for message passing • Level 0 Issues • What are PVM and MPI? • MPI Implementations • Inner-workings of PVM

  3. But First … • Please ask questions at any time  • Things will be more interesting when you do • I’d rather answer questions. • Got it?

  4. What is Message Passing? Why Do I Care • Message passing allows two processes to: • Exchange information • Synchronize with each other • Message passing is “Sockets for Dummies” • So? • Applications need much more power and or memory than a single machine can deliver • Large parallel programs need well-defined mechanisms to coordinate and exchange info

  5. Message Passing in the HPC World • Large scientific applications scale to 100’s of processors (routinely) and 1000’s of processors (in rare cases) • Climate/Ocean modeling • Molecular physics (QCD, dynamics, materials, …) • Computational Fluid Dynamics • And many more … • Message passing and SPMD programming style have been key infrastructure enablers • Why not shared memory?

  6. How Does Message Passing Differ from Socket Programming • Socket programming (OS 101) is a type of message passing • Open, bind, connect, accept too arcane • sendto, recvfrom (UDP) not reliable • Good point-to-point, multicast, broadcast are limited • Message passing usually means (pt-2-pt) • Low latency • High performance • Reliable, in-sequence delivery • Group operations • Broadcast • Reduce (eg, sum an array whose parts are held in different processes) • Group synchronize (barrier)

  7. Process A Initialize Send(B, “Hello World”) Recv(B, String) Print String “Hi There” Finalize Process B Initialize Recv(A, String) Print String “Hello World” Send(A, “Hi There”) Finalize Hello World – MP Style

  8. Message Addressing • Identify an endpoint • Use a tag to distinguish a particular message • pvm_send(dest, tag) • MPI_SEND(COMM, dest, tag, buf, len, type) • Receiving • recv(src, tag); recv(*,tag); recv (src, *); recv(*,*); • What if you want to build a library that uses message passing? Is (src, tag) safe in all instances?

  9. Level O Issues • Basic Pt-2-Pt Message Passing is straightforward, but how does one … • Make it go fast • Eliminate extra memory copies • Take advantage of specialized hardware • Move complex data structures (packing) • Receive from one-of-many (wildcarding) • Synchronize a group of tasks • Recover from errors • Start tasks • Build safe libraries • Monitor tasks • …

  10. MPI-1 addresses many of the level 0 issues (but not all)

  11. A long history of research efforts in message passing • P4 • Chameleon • Parmacs • TCGMSG • CHIMP • NX (Intel i860, Paragon) • PVM • … And these begot MPI

  12. So What is MPI • It is a standard message passing API • Specifies many variants of send/recv • 9 send interface calls • Eg., synchronous send, asynchronous send, ready send, asynchronous ready send • Plus other defined APIs • Process topologies • Group operations • Derived Data types • Profiling API (standard way to instrument MPI code) • Implemented and optimized by machine vendors • Should you use it? Absolutely!

  13. So What’s Missing in MPI-1? • Process control • How do you start 100 tasks? • How do you kill/signal/monitor remote tasks • I/O • Addressed in MPI-2 • Fault-tolerance • One MPI process dies, the rest eventually hang • Interoperability • No standard set for running a single parallel job across architectures (eg. Cannot split computation between x86 Linux and Alpha)

  14. What is PVM? Heterogeneous Virtual Machine support for: • Resource Management • add/delete hosts from a virtual machine • Process Control • spawn/kill tasks dynamically • Message Passing • blocking send, blocking and non-blocking receive, mcast • Dynamic Task Groups • task can join or leave a group at any time • Fault Tolerance • VM automatically detects faults and adjusts

  15. Popular PVM Uses • Poor man’s Supercomputer • Beowulf (PC) clusters, Linux, Solaris, NT • Cobble together whatever resources you can get • Metacomputer linking multiple Supercomputers • ultimate performance: eg. have combined nearly 3000 processors and up to 53 supercomputers • Education Tool • teaching parallel programming • academic and thesis research

  16. PVM In a Nutshell • Each host (could be an MPP or SMP) runs a PVMD • A collection of PVMDs define a virtual machine • Once configured, tasks can be started (spawned), killed, signaled from a console • Basic message passing • Performance is OK, But API Semantics limit optimizations

  17. MPI Design Goals • Make it go as fast as possible • Operate in a serverless (daemonless environment) • Specify portability but not interoperability • Standardize best practices of research environments • Encourage competing implementations • Enable the building of safe libraries • The “assembly language” of Message Passing

  18. MPI in the Marketplace • MPICH Mississippi-Argonne open source • A top-quality reference implementation • http://www-unix.mcs.anl.gov/mpi/mpich/ • High Performance Cluster MPIs • AM-MPI, FM-MPI, PM-MPI, GM-MPI, BIP-MPI • 10us latency, 100MB/sec on Myrinet • Vendor supported MPI • SGI, Cray, IBM, Fujitsu, Sun, Hitachi, … • MPI Vendors • ScaMPI, MPI Soft-Tech, Genias, …

  19. interoperability fault tolerance heterogeneity resource control dynamic model MPP performance many communication methods topology static model (SPMD) Comparisons Each API has its unique strengths PVM MPI Best Distributed Computing Best Large Multiprocessor Evaluate the needs of your application then choose

  20. PVM? MPI? • PVM is easy to use, especially on a network of workstations. Its message passing API is relatively simple • MPI is a standard, has a steeper learning curve and doesn’t have a standard way to start tasks • MPICH does have an “mpirun” command • If building a new scalable, production code, should use MPI (widely supported now) • If experimenting with message passing, are interested in dynamics, use PVM.

  21. Some Inner Workings of PVM • Every process has a unique, virtual-machine-wide, identifier called a task ID (TID) • PVMDs run on each host and act as points of presence • A single master PVMD disseminates current virtual machine configuration and holds something called the PVM mailbox. • The VM can grow and shrink around the master (if the master dies, the machine falls apart) • Dynamic configuration is used whenever practical

  22. pvmd - one PVM daemon per host host (one per IP address) task task task Unix Domain Sockets inner host messages tcp pvmd direct connect OS network interface pvmds fully connected using UDP task task task task task task task task task pvmd P0 P1 P2 task task task internal interconnect pvmd Shared Memory distributed memory MPP shared memory multiprocessor How PVM is Designed libpvm - task linked to PVM library

  23. PVM Tasks Can Use Multiple Transports • Uses sockets mostly • Unix-domain on host • TCP between tasks on different hosts • UDP between Daemons (custom reliability) • SysV Shared Memory Transport for SMPs • Tasks still use pvm_send(), pvm_recv() • Native MPP • PVM can ride atop a native MPI implementation

  24. Task ID (tid) • PVM uses tid to identify pvmd, tasks, groups • Fits into 32-bit integer • S bit addresses pvmd, G bit forms mcast address • Local part defined by each pvmd - eg. for PGON 12 bits 18 bits S G host ID local part 12 bits 7 bits 11 bits S G host ID process node ID 4096 hosts each with 2048 nodes

  25. Things to note about PVM Addressing • Addresses contain routing information by virtue of the host part • Transport selection at runtime is simplified • Bit-mask + table lookup • Moving a PVM task is very difficult • Condor (U. Wisc) with effort • Group/multicast bit makes it straightforward to implement multicast within pt-2-pt infrastructure

  26. Communication Context in MPI • MPI Wraps together Group and Context into a single entity called a Communicator • MPI program starts with one Communicator • MPI_COMM_WORLD • All communicators are derived from this • Library implementers are passed a communicator (group) and derive a new communicator -> Safe comm envelope • Messages have a 3-tuple to identify them • (comm, src, tag) • Comm cannot be wildcarded

  27. One task gets and distributes a new globally unique context newcontext = pvm_newcontext( ); broadcast newcontext to all tasks or put it in persistent message All tasks switch to safe context oldcontext = pvm_setcontext( newcontext)); Safe communication for your application or library Be aware: Unlike MPI, the current Context is not explicit in the Send/recv API newcontext = pvm_setcontext( oldcontext)); pvm_freecontext( newcontext); Communication Context in PVM

  28. Receiving a message (library viewpoint) • Messages arrive into a process and must be discriminated • Message header contains, src, tag, context, length, flags • Library “buffers” incoming messages until task receives • Must be match available messages with match criteria • Tasks may ask to process messages in a different order than they are actually received • (MPI has many variants of send/recv to handle various cases for optimization) • PVM allows message handlers to that when a particular match criteria occurs, a subroutine is called

  29. Handler function Incoming mesg. Active mesg. Data or Control messages Message Handlers Source,tag,context VM control messages User defined handlers pvmd

  30. Persistent Messages • Tasks can store and retrieve messages by name • Distributed information database for • dynamic programs • provides rendezvous, attachment, groups, many uses. • Multiple messages per “name” possible index = pvm_putinfo( name, msgbuf, flag) pvm_recvinfo( name, index, flag ) pvm_delinfo( name, index, flag ) pvm_getmboxinfo( pattern, #names, array of struct )

  31. Task 2 Task stores information eg. How to contact application, or Network load forecast, etc. Task 1 Later, another task can request this message and receive it normally Persistent Messages Task can specify when and who can replace a message it has placed in the message box. Message box Key: message Message box storage is coordinated across pvmds

  32. Monitoring Performance • PVM allows messages to be “traced” so that flows can be debugged • MPI provides a standard profiling interface to build profiling tools • Nupshot, Jumpshot, MPITrace, VaMPIr, … • XPVM (screen shot next slide) provides visual information about machine utilization, flows, configuration

  33. XPVM Screen Shot

  34. Wrapping Up • MPI has a very rich messaging interface and designed for efficiency • http://www-unix.mcs.anl.gov/mpi • PVM has a simple messaging interface + • Process control, Interoperability, Dynamics • http://www.epm.ornl.gov/pvm • Perform comparably when on Ethernet • MPI outperforms when on MPP • Both are still popular, but MPI is an accepted community standard with many support chains.

  35. Questions?

More Related