1 / 65

Course Outline

Course Outline. Introduction in algorithms and applications Parallel machines and architectures Overview of parallel machines, trends in top-500, clusters Programming methods, languages, and environments Message passing (SR, MPI, Java) Higher-level language: HPF Applications

clark
Download Presentation

Course Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Course Outline • Introduction in algorithms and applications • Parallel machines and architectures Overview of parallel machines, trends in top-500, clusters • Programming methods, languages, and environments Message passing (SR, MPI, Java) Higher-level language: HPF • Applications N-body problems, search algorithms • Grid computing Multimedia content analysis on Grids (guest lecture Frank Seinstra) • Many-core (GPU, Cell) programming (guest lectures Ana Varbanescu and Rob van Nieuwpoort)

  2. Parallel Programming on Computational Grids

  3. Outline • Grids • Parallel programming on grids • Case study: Ibis Paper: • Real-World Distributed Computing with Ibis,

  4. IEEE Computer Aug. 2010

  5. IEEE Computer Aug. 2010

  6. Grids • Seamless integration of geographically distributed computers,databases, instruments • The name is an analogy with power grids • Highly active research area • Open Grid Forum • Globus middleware • Many European projects, e.g.: • Gridlab: Grid Application Toolkit and Testbed • DEISA: Distributed European Infrastructure for Supercomputing Applications • XtreemOS: Linux-based OS for grids • Contrail (1 Oct 2010): Cloud computing • VL-e (Virtual laboratory for e-Science) project • ….

  7. Why Grids? • New distributed applications that use data or instruments across multiple administrative domains and that need much CPU power • Computer-enhanced instruments • Collaborative engineering • Browsing of remote datasets • Use of remote software • Data-intensive computing • Very large-scale simulation • Large-scale parameter studies

  8. Web, Grids, Clouds and e-Science • Web is about exchanging information • Grid is about sharing resources • Computers, data bases, instruments • Clouds are about hiring resources • Pay-as-you go, virtualization • e-Science supports experimental science by providing a virtual laboratory on top of Grids & Clouds • Support for visualization, workflows, data management,security, authentication, high-performance computing

  9. Grids Harness distributed resources The big picture Application Application Application Potential Generic part Potential Generic part Potential Generic part Management of comm. & computing Virtual Laboratory Application oriented services Management of comm. & computing Management of comm. & computing

  10. The data explosion • e-Science experiments generate much data, that often is distributed and that need much (parallel) processing • high-resolution imaging: ~ 1 GByte per measurement • Bio-informatics queries: 500 GByte per database • Satellite world imagery: ~ 5 TByte/year • Current particle physics: 1 PByte per year • LHC physics: 10-30 PByte per year

  11. Distributed supercomputing • Parallel processing on geographically distributed computing systems (grids) • Examples: • SETI@home (), RSA-155, Entropia, Cactus • Mostly limited to trivially parallel applications • Questions: • Can we generalize this to more HPC applications? • What high-level programming support is needed?

  12. Speedups on a grid? • Grids usually are hierarchical • Collections of clusters, supercomputers • Fast local links, slow wide-area links • Can optimize algorithms to exploit this hierarchy • Minimize wide-area communication • Wide-area bandwidth is increasing • DAS-3/DAS-4 have 10 Gb/s dedicated optical links between the sites • Wide-area latency remains high (limited by speed-of-light)

  13. Example: N-body simulation • Much wide-area communication • Each node needs info about remote bodies CPU 1 CPU 1 CPU 2 CPU 2 Amsterdam Delft

  14. Trivial optimization CPU 1 CPU 1 CPU 2 CPU 2 Amsterdam Delft

  15. Wide-area optimizations • Message combining on wide-area links • Latency hiding on wide-area links • Collective operations for wide-area systems • Broadcast, reduction, all-to-all exchange • Load balancing Conclusions: • Many applications can be optimized to run efficiently on a hierarchical wide-area system • Need better programming support

  16. Outline • Grids • Case study: Ibis • Paper: • Real-World Distributed Computing with Ibis, Sept. 2003

  17. The Ibis system • High-level & efficient programming support for distributed supercomputing • Use Java-centric approach + JVM technology • Inherently more portable than native compilation • Goal: drastically simplify programming and deployment of high performance distributed applications • Target: • Large-scale distributed systems, including clusters, grids, desktop grids, clouds, mobile devices …. • Possibly all at the same time for 1 application

  18. Real-world distributed systems

  19. World wide testbed

  20. Problem • How to write (high-performance) applications for real-world distributed systems? • How to deal with: • Performance: efficiency on wide-area system • Heterogeneity: different systems & APIs • Malleability: resources come and go • Fault tolerance: crashes • Connectivity: firewalls, NAT, etc.

  21. Our approach • Study fundamental underlying problems • … hand-in-hand with realistic applications • … integrate solutions in one system: Ibis ! User Distributed Systems

  22. Applications • Scientific applications • Imaging (VU Medical Center, AMOLF) • Bioinformatics (sequence analysis) • Astronomy (data analysis challenge) • Multimedia content analysis • Games and model checking • Semantic web (distributed reasoning)

  23. Multimedia content analysis • Automatically extract information from images & video • E.g., video archive, surveillance cameras • Extract feature vectors from images • Describe properties (color, shape) • Data-parallel task on a cluster • Compute on consecutive images • Task-parallelism on a grid

  24. Example: object recognition • Analyze video stream from camera to learn and recognize every-day objects • Representative for more serious applications • Same algorithms used for surveillance cameras • London Underground  >120.000 years of processing for >> 10.000’s CCTV cameras

  25. Games and Model Checking • Cansolve entire Awari game onwide-area DAS-3 (889 B positions) • Needs 10G private optical network • Distributed model checking has verysimilar communication pattern • Search huge state spaces, random work distribution, bulk asynchronous transfers • Can efficiently run DeVinE model checker on wide-area DAS-3, use up to 1 TB memory

  26. Distributed reasoning • MaRVIN (Frank van Harmelen et al, VU): • A distributed platform for massive RDF inferencing (deductive closure) • ``a brain the size of a planet’’ • Uses Ibis to run on heterogeneous systems (clusters, desktop grids) • Used for Billion Triple track of Semantic Web Challenge 2008 • Inputs 800M RDF triples, derives 29B triples

  27. WebPIEWeb-scale Parallel Inference Engine • MapReduce-based distributed RDFS/OWL inference engine using Hadoop (not yet Ibis) • Used up to 100 Billion triples • Won CCGrid SCALE2010 Award

  28. Awards Astronomy DACH 2008 – BS DACH 2008 - FT (Cluster/Grid’08) SCALE 2008 (CCGrid’08) ISWC 2008 / SCALE 2010 Multimedia Computing Semantic Web (van Harmelen et al.) AAAI-VC 2007

  29. Ibis Philosophy • Real-world distributed applications should be developed and compiled on a local workstation, and simply be launched from there

  30. Ibis Approach • Virtual Machines (Java) deal with heterogeneity • Provide range of programming abstractions • Designed for dynamic/faulty environments • Easy deployment through middleware-independent programming interfaces • Modular and flexible: can replace Ibis components by external ones

  31. Programming Logical Likes math Deployment Practical Visual (GUI) Ibis Design • Applications need functionality for • Programming (as in programming languages) • Deployment (as in operating systems)

  32. Ibis System

  33. Ibis brains

  34. Programming system

  35. Programming models • Message passing (IPL, RMI, MPJ) • Satin: • Fault-tolerant, malleable divide-and-conquer system • Jorus: • Transparent library with multimedia operations • Maestro: • Self-optimizing fault-tolerant dataflow framework

  36. Satin: a parallel divide-and-conquer system on top of Ibis • Divide-and-conquer is inherently hierarchical • More general than master/worker • Satin: Cilk-like primitives (spawn/sync) in Java

  37. Example interface FibInter { public int fib(long n); } class Fib implements FibInter { int fib (int n) { if (n < 2) return n; return fib(n-1) + fib(n-2); } } Single-threaded Java

  38. Example interface FibInter extends ibis.satin.Spawnable { public int fib(long n); } class Fib extends ibis.satin.SatinObject implements FibInter { public int fib (int n) { if (n < 2) return n; int x = fib (n - 1); int y = fib (n - 2); sync(); return x + y; } } Java + divide&conquer

  39. IPL (Ibis Portability Layer) • Java-centric “run-anywhere” library • Point-to-point, multicast, streaming • Simple model for tracking resources • Join-Elect-Leave • Supports malleability & fault-tolerance

  40. SmartSockets library • Detects connectivity problems • Tries to solve them automatically • With as little help from the user as possible • Integrates existing and several new solutions • Reverse connection setup, STUN, TCP splicing, SSH tunneling, smart addressing, etc. • Uses network of hubs as a side channel

  41. SmartSockets

  42. Ibis Deployment system

  43. IbisDeploy GUI

  44. JavaGAT • GAT: Grid Application Toolkit • Makes grid applications independent of the underlying grid infrastructure • Used by applications to access grid services • File copying, resource discovery, job submission & monitoring, user authentication • Successor API is currently being standardized

  45. Grid Applications with GAT Grid Application File.copy(...)‏ submitJob(...)‏ GAT Remote Files Monitoring Info service Resource Management GAT Engine GridLab Globus Unicore SSH P2P Local Intelligentdispatching globus gridftp Koala

  46. JavaGat example

  47. Zorilla: Java P2P supercomputing middleware

  48. Ibis Architecture

  49. Ibis demo (movie)

  50. Object recognition • Runs simultaneously on clusters (DAS-3, Japan, Australia), Desktop Grid, Amazon EC2 Cloud • Connectivity problems solved automatically by Ibis SmartSockets Ibis (Java) Client Servers Broker

More Related