650 likes | 772 Views
Course Outline. Introduction in algorithms and applications Parallel machines and architectures Overview of parallel machines, trends in top-500, clusters Programming methods, languages, and environments Message passing (SR, MPI, Java) Higher-level language: HPF Applications
E N D
Course Outline • Introduction in algorithms and applications • Parallel machines and architectures Overview of parallel machines, trends in top-500, clusters • Programming methods, languages, and environments Message passing (SR, MPI, Java) Higher-level language: HPF • Applications N-body problems, search algorithms • Grid computing Multimedia content analysis on Grids (guest lecture Frank Seinstra) • Many-core (GPU, Cell) programming (guest lectures Ana Varbanescu and Rob van Nieuwpoort)
Outline • Grids • Parallel programming on grids • Case study: Ibis Paper: • Real-World Distributed Computing with Ibis,
Grids • Seamless integration of geographically distributed computers,databases, instruments • The name is an analogy with power grids • Highly active research area • Open Grid Forum • Globus middleware • Many European projects, e.g.: • Gridlab: Grid Application Toolkit and Testbed • DEISA: Distributed European Infrastructure for Supercomputing Applications • XtreemOS: Linux-based OS for grids • Contrail (1 Oct 2010): Cloud computing • VL-e (Virtual laboratory for e-Science) project • ….
Why Grids? • New distributed applications that use data or instruments across multiple administrative domains and that need much CPU power • Computer-enhanced instruments • Collaborative engineering • Browsing of remote datasets • Use of remote software • Data-intensive computing • Very large-scale simulation • Large-scale parameter studies
Web, Grids, Clouds and e-Science • Web is about exchanging information • Grid is about sharing resources • Computers, data bases, instruments • Clouds are about hiring resources • Pay-as-you go, virtualization • e-Science supports experimental science by providing a virtual laboratory on top of Grids & Clouds • Support for visualization, workflows, data management,security, authentication, high-performance computing
Grids Harness distributed resources The big picture Application Application Application Potential Generic part Potential Generic part Potential Generic part Management of comm. & computing Virtual Laboratory Application oriented services Management of comm. & computing Management of comm. & computing
The data explosion • e-Science experiments generate much data, that often is distributed and that need much (parallel) processing • high-resolution imaging: ~ 1 GByte per measurement • Bio-informatics queries: 500 GByte per database • Satellite world imagery: ~ 5 TByte/year • Current particle physics: 1 PByte per year • LHC physics: 10-30 PByte per year
Distributed supercomputing • Parallel processing on geographically distributed computing systems (grids) • Examples: • SETI@home (), RSA-155, Entropia, Cactus • Mostly limited to trivially parallel applications • Questions: • Can we generalize this to more HPC applications? • What high-level programming support is needed?
Speedups on a grid? • Grids usually are hierarchical • Collections of clusters, supercomputers • Fast local links, slow wide-area links • Can optimize algorithms to exploit this hierarchy • Minimize wide-area communication • Wide-area bandwidth is increasing • DAS-3/DAS-4 have 10 Gb/s dedicated optical links between the sites • Wide-area latency remains high (limited by speed-of-light)
Example: N-body simulation • Much wide-area communication • Each node needs info about remote bodies CPU 1 CPU 1 CPU 2 CPU 2 Amsterdam Delft
Trivial optimization CPU 1 CPU 1 CPU 2 CPU 2 Amsterdam Delft
Wide-area optimizations • Message combining on wide-area links • Latency hiding on wide-area links • Collective operations for wide-area systems • Broadcast, reduction, all-to-all exchange • Load balancing Conclusions: • Many applications can be optimized to run efficiently on a hierarchical wide-area system • Need better programming support
Outline • Grids • Case study: Ibis • Paper: • Real-World Distributed Computing with Ibis, Sept. 2003
The Ibis system • High-level & efficient programming support for distributed supercomputing • Use Java-centric approach + JVM technology • Inherently more portable than native compilation • Goal: drastically simplify programming and deployment of high performance distributed applications • Target: • Large-scale distributed systems, including clusters, grids, desktop grids, clouds, mobile devices …. • Possibly all at the same time for 1 application
Problem • How to write (high-performance) applications for real-world distributed systems? • How to deal with: • Performance: efficiency on wide-area system • Heterogeneity: different systems & APIs • Malleability: resources come and go • Fault tolerance: crashes • Connectivity: firewalls, NAT, etc.
Our approach • Study fundamental underlying problems • … hand-in-hand with realistic applications • … integrate solutions in one system: Ibis ! User Distributed Systems
Applications • Scientific applications • Imaging (VU Medical Center, AMOLF) • Bioinformatics (sequence analysis) • Astronomy (data analysis challenge) • Multimedia content analysis • Games and model checking • Semantic web (distributed reasoning)
Multimedia content analysis • Automatically extract information from images & video • E.g., video archive, surveillance cameras • Extract feature vectors from images • Describe properties (color, shape) • Data-parallel task on a cluster • Compute on consecutive images • Task-parallelism on a grid
Example: object recognition • Analyze video stream from camera to learn and recognize every-day objects • Representative for more serious applications • Same algorithms used for surveillance cameras • London Underground >120.000 years of processing for >> 10.000’s CCTV cameras
Games and Model Checking • Cansolve entire Awari game onwide-area DAS-3 (889 B positions) • Needs 10G private optical network • Distributed model checking has verysimilar communication pattern • Search huge state spaces, random work distribution, bulk asynchronous transfers • Can efficiently run DeVinE model checker on wide-area DAS-3, use up to 1 TB memory
Distributed reasoning • MaRVIN (Frank van Harmelen et al, VU): • A distributed platform for massive RDF inferencing (deductive closure) • ``a brain the size of a planet’’ • Uses Ibis to run on heterogeneous systems (clusters, desktop grids) • Used for Billion Triple track of Semantic Web Challenge 2008 • Inputs 800M RDF triples, derives 29B triples
WebPIEWeb-scale Parallel Inference Engine • MapReduce-based distributed RDFS/OWL inference engine using Hadoop (not yet Ibis) • Used up to 100 Billion triples • Won CCGrid SCALE2010 Award
Awards Astronomy DACH 2008 – BS DACH 2008 - FT (Cluster/Grid’08) SCALE 2008 (CCGrid’08) ISWC 2008 / SCALE 2010 Multimedia Computing Semantic Web (van Harmelen et al.) AAAI-VC 2007
Ibis Philosophy • Real-world distributed applications should be developed and compiled on a local workstation, and simply be launched from there
Ibis Approach • Virtual Machines (Java) deal with heterogeneity • Provide range of programming abstractions • Designed for dynamic/faulty environments • Easy deployment through middleware-independent programming interfaces • Modular and flexible: can replace Ibis components by external ones
Programming Logical Likes math Deployment Practical Visual (GUI) Ibis Design • Applications need functionality for • Programming (as in programming languages) • Deployment (as in operating systems)
Programming models • Message passing (IPL, RMI, MPJ) • Satin: • Fault-tolerant, malleable divide-and-conquer system • Jorus: • Transparent library with multimedia operations • Maestro: • Self-optimizing fault-tolerant dataflow framework
Satin: a parallel divide-and-conquer system on top of Ibis • Divide-and-conquer is inherently hierarchical • More general than master/worker • Satin: Cilk-like primitives (spawn/sync) in Java
Example interface FibInter { public int fib(long n); } class Fib implements FibInter { int fib (int n) { if (n < 2) return n; return fib(n-1) + fib(n-2); } } Single-threaded Java
Example interface FibInter extends ibis.satin.Spawnable { public int fib(long n); } class Fib extends ibis.satin.SatinObject implements FibInter { public int fib (int n) { if (n < 2) return n; int x = fib (n - 1); int y = fib (n - 2); sync(); return x + y; } } Java + divide&conquer
IPL (Ibis Portability Layer) • Java-centric “run-anywhere” library • Point-to-point, multicast, streaming • Simple model for tracking resources • Join-Elect-Leave • Supports malleability & fault-tolerance
SmartSockets library • Detects connectivity problems • Tries to solve them automatically • With as little help from the user as possible • Integrates existing and several new solutions • Reverse connection setup, STUN, TCP splicing, SSH tunneling, smart addressing, etc. • Uses network of hubs as a side channel
JavaGAT • GAT: Grid Application Toolkit • Makes grid applications independent of the underlying grid infrastructure • Used by applications to access grid services • File copying, resource discovery, job submission & monitoring, user authentication • Successor API is currently being standardized
Grid Applications with GAT Grid Application File.copy(...) submitJob(...) GAT Remote Files Monitoring Info service Resource Management GAT Engine GridLab Globus Unicore SSH P2P Local Intelligentdispatching globus gridftp Koala
Object recognition • Runs simultaneously on clusters (DAS-3, Japan, Australia), Desktop Grid, Amazon EC2 Cloud • Connectivity problems solved automatically by Ibis SmartSockets Ibis (Java) Client Servers Broker