1 / 21

Integrated Synchronization and Consistency Protocol for Parallel Programming

This research paper presents an integrated synchronization and consistency protocol for the implementation of a high-level parallel programming language. The protocol combines synchronization and consistency protocols to ensure efficient and deterministic execution of parallel programs. The paper also provides experimental results comparing different consistency mechanisms.

archibald
Download Presentation

Integrated Synchronization and Consistency Protocol for Parallel Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Integrated Synchronization and Consistency Protocol for the Implementation of a High-Level Parallel Programming Language Martin C. Rinard University of California, Santa Barbara Santa Barbara, California 93106

  2. Context • Parallel Programming System • Single Address Space Programming Model • Message Passing Machine • Key Functionality • Synchronization • Communication • Consistency • Straightforward Implementation Separates Protocols • Synchronization Messages • Communication and Consistency Messages • Redundant Messages

  3. Jade High-Level, Implicitly Parallel Language • Single Address Space • Serial Semantics • Portable • Shared Memory Machines - Stanford DASH • Message Passing Machines - iPSC/860, CM-5 • Heterogeneous Networks of Workstations • Message Passing Implementation • Combines Synchronization and Consistency Protocols • Result: Single Efficient Protocol

  4. Structure of Talk • Jade Programming Model • Synchronization Mechanism • Integrated Synchronization and Consistency Protocol • Experimental Results

  5. Programming Model • Programmer • Starts With A Serial Program • Identifies Data Granularity using Shared Objects • Identifies Computation Granularity using Tasks • Provides Access Specification For Each Task • Jade Implementation Analyzes Access Specifications • Automatically Extract Parallelism • Synchronize Computation • Generate Communication • Ensure Consistency of Shared Objects • Preserves Serial Semantics

  6. Programming Model Starts With Serial Program Data Granularity - Shared Objects Task Granularity Data Usage Information Extracts Concurrency Synchronizes Computation Generates Communication Ensures Consistency of Shared Objects Programmer Implementation

  7. rd rd wr wr rd rd wr wr wr wr Concurrency Principle • Tasks Execute Serially if • One declares a write to a piece of data • Other declares either read or write to the same piece of data • Same execution order as serial program • Otherwise, Tasks Execute Concurrently wr

  8. Key Aspects • Implementation Manages Concurrency • Synchronization • Communication • Consistency • Promotes Modularity • Serial Semantics • Deterministic Execution • Only Serial Programs • Portable

  9. rd rd wr wr wr Object Queues wr rd rd wr wr

  10. rd rd wr wr wr rd rd wr wr wr rd rd wr wr Object Queues wr rd rd wr rd rd Time rd rd

  11. wr rd rd wr wr Consistency Problem • Invalidate Protocol • Update Protocol Processor 0 Processor 1 Time Expensive Global Operations

  12. Consistency Mechanism • Object Replica Has Version Number • Object Queue Maintains • Latest Version Number • Owner of Latest Version • Queue Tells Each Task • What Version to Access • Owner of that Version • Enabled Task Checks Version Numbers • If Obsolete, Fetch From Owner • If Up to Date, No Fetch

  13. rd rd wr wr wr 2 2 wr 2 3 rd rd 2 3 wr wr rd rd 3 3 wr wr Consistency Mechanism Processor 0 Processor 1 2 wr rd rd rd rd wr wr 3 rd rd 3 rd rd 3 rd rd

  14. Result • A More Efficient Consistency Mechanism • No Required Invalidate Messages • No Required Update Messages • Enabling Features • High-Level Language Model • Data Usage Information • Integration With Synchronization Mechanism

  15. Experimental Results • Compare Three Consistency Mechanisms • Update (Broadcast if all Processors should receive Object) • Invalidate • Version • Applications • Panel Cholesky • Ocean • Water • Collected Data on iPSC/860 • Communication Volume • Number of Messages

  16. Panel Cholesky Number of Messages Communication Volume

  17. Ocean Number of Messages Communication Volume

  18. Water Number of Messages Communication Volume

  19. Adaptive Broadcast • Goal • Advantages of Broadcast for Widely Accessed Data • Without Disadvantages of Update Protocol • Policy • If Every Processor Reads Same Version of Object • Jade Implementation Uses Broadcast Protocol for that Object • Result • Performs as well as Update Protocol for Water • Performs as well as Version for Panel Cholesky and Ocean

  20. Related Work • Consistency Based on Version Numbers • NAMOS (Reed) • Sprite (Nelson, Welch, Ousterhout) • SAM (Scales, Lam) • VDOM (Feeley, Levy) • TreadMarks (Keleher, Dwarkadas, Cox, Zwaenepoel) • Midway (Zekauskas, Sawdon, Bershad)

  21. Conclusion • Jade Presents High-Level Abstractions • Serial Semantics • Single Address Space • Jade Implementation Provides an Integrated Protocol • Synchronization • Communication • Consistency • Experimental Results • Adaptive Broadcast

More Related