1 / 17

Models and Languages for Parallel Computation

Models and Languages for Parallel Computation. D. Skillicorn and D. Talia, ACM Computing Surveys, Vol. 30, No. 2, June 1998. Presented by Russell Zuck June 28, 2005. Overview. Motivation Difficulties of parallel programming Evaluation criteria Categories of parallel programming models

illias
Download Presentation

Models and Languages for Parallel Computation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Models and Languages for Parallel Computation D. Skillicorn and D. Talia, ACM Computing Surveys, Vol. 30, No. 2, June 1998 Presented by Russell Zuck June 28, 2005

  2. Overview • Motivation • Difficulties of parallel programming • Evaluation criteria • Categories of parallel programming models • Summary of trends in model development • A related perspective and observation

  3. Motivation • What programming models are out there? • How well do they help solve some of the problems inherent with parallel programming? • How difficult are they to use? • What type of architecture is it used on?

  4. Difficulties of Parallel Programming • Many architectures • Most are special purpose • No standards • Many languages/tools • Optimizers don’t work well • Wide range of programming paradigms • Lack of skilled programmers • Parallel thinking • Lack of a substantial market

  5. Evaluation Criteria • Easy to program • Decomposition • Mapping • Communication • Synchronization • Software Development Methodology • How do you determine correctness? • Sequential techniques of testing and debugging don’t extend well to parallel systems • Large state space to test due to extra degree of freedom • Testing limited to a couple of architectures

  6. Evaluation Criteria • Architecture independence • Easy to understand • Can a large number of people become proficient at it? • Guaranteed performance • How much will execution performance differ from one architecture to another? • Cost Measures • Execution time • Processor utilization • Development costs

  7. Parallel Programming Models • Differ in the degree of abstraction of concepts from the programmer • Parallelism • Decomposition • Mapping • Communication • Synchronization • The ideal model • The programmer is not required to be explicitly involved in any of the above operations

  8. Nothing Explicit • Complete abstraction from parallel operations • These exist only as “we are nearly there” types of languages • Examples • Optimizing compilers • Haskell: High order functional language

  9. Parallelism Explicit • Techniques for identifying parallelism required • Library functions • Syntax extensions • Example • Fortran

  10. Decomposition Explicit • Programmer must specify the division of the problem into parallelizable pieces • Library functions • Example • BSP (Bulk synchronous parallelism)

  11. Mapping Explicit • Programmer must specify the distribution of program pieces • Example • RPC • Paralf: An annotated functional language

  12. Communication Explicit • Programmer responsible for interprocessor communication • Example • ABCL/1

  13. Everything Explicit • Almost no abstraction from the programmer • Examples • Orca • PRAM • MPI

  14. Summary of Trends • So which is the best compromise? • As with most things, the middle of the road seems to be the best • A model with a medium amount of programmer abstraction • Trends in model development • Work on models with little abstraction is diminishing • Concentration of effort on models with midrange abstraction. Conceal some aspects of parallelism while preserving expressiveness • Some hope still resides with highly abstract models…Don’t Hold Your Breath Too Long!!!

  15. Another Perspective • Speculation of the future • As parallel machines regain popularity, more manufactures will be willing to produce them • Eventually a handful of standard architectures will emerge to cover SIMD, MIMD (Shared Memory), and MIMD (Fixed Connection) • Alternate solution • Use a one or two development languages and develop virtual machines (middle-ware) for each type of architecture • Similar to the Java paradigm

  16. Programming Language VM1 VM2 VM3 MIMD (Shared) MIMD (Fixed) SIMD

More Related