1 / 6

Data Movement Mechanisms Malcolm Atkinson Director of National e-Science Centre www.nesc.ac.uk 1 st May 2003 IBM Almade

Data Movement Mechanisms Malcolm Atkinson Director of National e-Science Centre www.nesc.ac.uk 1 st May 2003 IBM Almaden Research Centre DAIS WG Face-to-Face. Motivation. Results of no value If not delivered to a consumer Updates Need supplied values These may be large!

kira
Download Presentation

Data Movement Mechanisms Malcolm Atkinson Director of National e-Science Centre www.nesc.ac.uk 1 st May 2003 IBM Almade

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Movement Mechanisms Malcolm Atkinson Director of National e-Science Centre www.nesc.ac.uk 1st May 2003 IBM Almaden Research Centre DAIS WG Face-to-Face

  2. Motivation • Results of no value • If not delivered to a consumer • Updates • Need supplied values • These may be large! • Supplier and Consumer(s) • Same process or different processes • Same machine or same cluster • Same LAN or Wide Area hop • Same architecture, hosting environment or different • Are messages transmitted from triggers always small? • Are they in or out of scope?

  3. Requirements • Information Preservation • A value transmitted arrives as the same value • For all values of all types • Composability • Any Grid (Data) Service with • Any Data Movement Mechanism • Performance & Costs • Choices to be supported?

  4. Requirements 2 • Integration • Available between all services • Available for all Information • Data, Code, Processes, State • DAIS-WG current focus is Information • Higher-level view of data • but future proof? • Control • What is moved (Query result|DB | File I set of (files | results), …) • Where From & To* (Notation in Request document?) • What units of transfer (Logical | Physical) • What synchronisation (and lifetime) • Privacy, e.g. Encryption • Dynamic selectivity & Flow control • What compression / encoding • What transformations en route • What monitoring mechanisms

  5. Fundamental Issues • What portType(s)? • What is (are) their functions? • What semantics of the functions? • Effects on data • Effects on GD* state • Reflection in SDEs • Failure modes

  6. Desiderata • Algebra of operations • B = Move (A); C = Move (B) • x = F(A), y = F(B)  x = y • Factories & Movement: Retained Relationships • Create a GD* service from A • Create a GD* service from A @ another loc • (Define) Shared state • Copied state • Mechanisms for copied state coherence • How up-to-date? • How is change propagated? • Retained relationships & Coherence mechanisms not currently in scope of DAIS standard?

More Related