1 / 22

Design of Cloud Management Layer for High-Performance File Transfer

Design of Cloud Management Layer for High-Performance File Transfer. 高效能檔案傳輸之雲端層設計. Outline. Introduction Background System Design Implementation Experiments. Introduction. Motivation File Transfer with load sharing and fault tolerance. Introduction.

bikita
Download Presentation

Design of Cloud Management Layer for High-Performance File Transfer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design of Cloud Management Layerfor High-Performance File Transfer 高效能檔案傳輸之雲端層設計

  2. Outline • Introduction • Background • System Design • Implementation • Experiments

  3. Introduction • Motivation • File Transfer with load sharing and fault tolerance.

  4. Introduction • Clients can upload the files to the cloud, and promise your file cannot be lose . Files Upload Download Files User (Client) Cluster Storage Server Management Layer

  5. Introduction (Cont.) • Surveillance Application Host Device x 30 2000G(2T) / 30 = 67G 67G / 0.3G = 223h 223h / 24h = 9.3 day Size: 0.2~0.5 GB/1hr Storage:2TB

  6. Introduction (Cont.) • Used Platforms and API • Hadoop • HadoopDistributed File System (HDFS) • File Storage • Hbase • User Authentication • Socket • File Data • Transmissions

  7. Outline • Introduction • Background • System Design • Implementation • Experiments

  8. Background (Cont.) What is Cloud Computing? • Scalable Computing and Storage Resource. Cloud Source Models

  9. Background (Cont.) Hadoop DataBase Parallel Processing File System • The other components of Hadoop

  10. Background (Cont.) What is “Hadoop Distributed File System (HDFS)”? Clients Transmission Cluster Storage Server PC Datanode Namenode

  11. Background (Cont.) • Hbasehave high availability, high performance, and high expansion flexibility.

  12. Outline • Introduction • Background • System Design • Implementation • Experiments • Conclusions • Reference

  13. System Design(Cont.) • Components • Manager Selection • Authentication • Synchronization • Cloud-Based File Transfer

  14. System Design(Cont.) • Manager Selection • Connect Management layer and perform load balance • Collect memory load information from each nodes • Select the lowest memory load as connection port(Nodei) • Connect with Nodei • Authentication

  15. System Design(Cont.) • Authentication • Key in account / password • Authentication(Hbaseverification) • Synchronization

  16. System Design(Cont.) • Synchronization • HDFS files synchronization in client side • Check files existence • Cloud-Based files transfer • Background Monitoring

  17. System Design(Cont.) • Synchronization • HDFS files synchronization in client side • Check files existence • Cloud-Based files transfer • Background Monitoring

  18. System Design(Cont.) • Cloud-Based File Transfer • Send request using socket • Analysis The packet (to extract the command and path) • Execute command (Write/Send/Delete)

  19. Outline • Introduction • Background • System Design • Experiments • Conclusions • Reference

  20. Experiments • Manager Selection • 隨機配置 • 負載平衡配置 • 選擇負載最小之節點：5 個 • 更新時間：10 秒 • 可同時連線數量：10 個 • Environmental Environment 1 • Servers: 50, 100, 150, 200, 250 • Clients: 2000 • Environmental Environment2 • Servers: 50 • Clients:1000, 2000, 3000, 4000, 5000

  21. Experiments(Cont.) • Experiment 1 • Servers: 50, 100, 150, 200, 250 • Clients: 2000

  22. Experiments(Cont.) • Experiment 2 • Servers: 50 • Clients:1000, 2000, 3000, 4000, 5000

More Related