1 / 5

The Armada framework for parallel I/O on computational grids

The Armada framework for parallel I/O on computational grids. Ron Oldfield and David Kotz Department of Computer Science Dartmouth College. Must use large remote datasets Often computationally intensive Datasets often need pre- and/or post-processing Examples Climate modeling (EOS-DAS)

veata
Download Presentation

The Armada framework for parallel I/O on computational grids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Armada framework for parallel I/O on computational grids Ron Oldfield and David Kotz Department of Computer Science Dartmouth College FAST 2002 WIP Presentation

  2. Must use large remote datasets Often computationally intensive Datasets often need pre- and/or post-processing Examples Climate modeling (EOS-DAS) Astronomy (Digital Sky Surveys) Comp. Biology (Computed MicroTomography) Computational physics Flexibility Application control of the interface Application control of system policies (caching, data-dist., …) Performance Parallel data transfers. Remote execution of user code (e.g., filtering, transforms, compression, encryption) Introduction Computational Grids: geographically distributed networks of heterogeneous computer systems and devices. Data-intensive grid applications I/O system requirements FAST 2002 WIP Presentation

  3. Flexible design, based on stackable file systems. Applications access data through a graph of “ships” called an “armada”. Requests travel toward data servers. Data is pushed toward clients for reads, pulled toward servers for writes. The armada abstracts details of the I/O system Caching, filtering, data distribution clients on site A API data flow added by application filter replica from data provider dist dist file file file file file data segments on site B data segments from site C Armada An I/O framework for data-intensive grid applications FAST 2002 WIP Presentation

  4. clients on site A clients on site A API API API API combine filter dist dist replica replica replica filter filter filter filter filter dist dist file file file file file file file file file file data segments on site B data segments from site C data segments on site B data segments from site C Improving Performance FAST 2002 WIP Presentation

  5. In progress… • Automate graph restructuring. • Formalize rules and algorithms • Develop placement algorithms. • Requires detailed information on ship requirements and available resources. • Performance monitoring and analysis. Contact Information Ron Oldfield (raoldfi@cs.dartmouth.edu) David Kotz (dfk@cs.dartmouth.edu) http://www.cs.dartmouth.edu/~dfk/armada FAST 2002 WIP Presentation

More Related