1 / 15

Cross-Layer Scheduling in Cloud Computing Systems

Cross-Layer Scheduling in Cloud Computing Systems. Authors: Hilfi Alkaff , Indranil Gupta. Motivation. Many cloud computing frameworks out there Batch Processing Framework: Hadoop Stream Processing Framework: Storm Current applications are not aware of underlying network topology

sora
Download Presentation

Cross-Layer Scheduling in Cloud Computing Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cross-Layer Scheduling in Cloud Computing Systems Authors: HilfiAlkaff, Indranil Gupta

  2. Motivation • Many cloud computing frameworks out there • Batch Processing Framework: Hadoop • Stream Processing Framework: Storm • Current applications are not aware of underlying network topology • Might schedule tasks on machines with low bandwidth.

  3. Challenges • Need to expose underlying network topology efficiently to applications • Huge state space to search • Thousands of machines in a cluster • Users demand more interactive jobs • Multiple possible data-path representation • Want to have generic schedulers

  4. Data-Path: Map-Reduce

  5. Data-Path: Stream

  6. Proposed Solution • Cross-Layer Scheduling Framework • First-level scheduler in application Level • Second-level scheduler in routing level • Use Simulated Annealing at each level • Probabilistic framework • Idea: If neighboring state is better, always move there but if it is not, move there with probability P(T) that decreases with time

  7. Proposed Architecture Application Master Cross-Layer Scheduling SDN Controller

  8. Algorithm: Pre-computation • Pre-compute all-pairs (, k-shortest paths • Stored in Topology-Map hash-table with key=(, , value=array of k-shortest paths • Too many duplicates • Intelligently merge similar sub-paths • Hash-Table’s value is now a tree instead of array

  9. Algorithm: Main

  10. Algorithm: genState() Heuristic • Too many neighboring states • Not possible to traverse all of them • Application Level • Prefer node that has higher # of sink vertices • Prefer node that has higher # of source vertices • Routing Level • Prefer paths that have lower number of hops • Prefer paths that have higher amount of available bandwidth

  11. Emulab Result: Throughput

  12. Simulation Result: Computation Time

  13. Simulation Results: CDF

  14. Le Questions?

  15. Algorithm: Failures • Link-Failures • Need to re-allocate flows using that link • Keep a separate hash-table where key=edge, value=flows • Get another path from Topology-Map. • Machine-failures • Re-run main algorithm on

More Related