1 / 11

Efficient Large-Scale File Distribution Algorithm for Operating Systems Presentation

Research presented by Chris Miller & Pramita Mitra on distributing large datasets efficiently over distributed networks. Utilizing CCL storage pool model for sequential and parallel file distribution to optimize network resources. Evaluation of Best Neighbor Approximation method for improved distribution speed.

virginiahudson
Download Presentation

Efficient Large-Scale File Distribution Algorithm for Operating Systems Presentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Large Scale File DistributionSequential Branching Distribution Final Presentation Grad Operating Systems Presented by Chris Miller & Pramita Mitra Dec 13, 2006

  2. Problem Statement • Research requires distribution of large datasets on distributed networks • Methods such as multicast are too complicated to implement reliably • Tools available for file distribution • Chirp • Parrot • Algorithm needed to efficiently schedule the distribution of files

  3. Solution • Using CCL storage pool as model of distributed network • Using small, measured steps to find what aspects of distribution work best in implementation • Sequential distribution • Parallel distribution Distributor Ineffiecient use of network resources. Total time for distribution O(n). Stage 1 … Stage n Stage 2 Distributor Node 1 Node 2 … Node n Total time for distribution O(n).

  4. Baseline Results

  5. Sequential Branching Distribution Nodeset Distributor Thirdput Thirdput Stage 2 Stage 1 Stage 3 Thirdput Stage 2 Stage 3 Stage 3 Stage 3 Total time for distribution O(log2 n)

  6. Best Neighbor Approximation

  7. Best Neighbor Approximation

  8. Probabilistic Weighted Average

  9. Best Neighbor Approximation

  10. Results

  11. Conclusions • A fast and reliable distribution method is possible with simple file transfer methods • Distribution system is fault tolerant for all nodes except distributor node • Latency measurement • moderate indicator of transfer rate • low overhead • Small file transfer approximation • strong indicator of transfer rate • high overhead • Performance is near O(log2 n)

More Related