230 likes | 336 Views
SCC Industry Track July 10, 2008 A Service-Oriented Approach to Storage Backup. Hao Cheng, Yao H. Ho, Kien A. Hua, Danzhou Liu, Fei Xie, and Ynn-Pyng Tsaur Data Systems Lab School of EECS University of Central Florida. Project Objectives. Low Cost. High Performance.
E N D
SCC Industry Track July 10, 2008A Service-Oriented Approach to Storage Backup Hao Cheng, Yao H. Ho, Kien A. Hua, Danzhou Liu, Fei Xie, and Ynn-Pyng Tsaur Data Systems Lab School of EECS University of Central Florida
Project Objectives Low Cost High Performance Fault Tolerance
Total Cost of Ownership • A backup solution incurs both direct and indirect costs
Lower Total Cost of Ownership • Remove the hardware cost
Lower Total Cost of Ownership • Significantly reduce the operating cost • Maybe, customers will be willing to pay more for the software ?
The Challenges How about leveraging on existing hardware ??? Remove the hardware Significantly reduce the operation cost
Fact 1: Client Server Model • Most applications are based on client-server model. Server Network
Fact 2: Having Own Servers • Business units have their own servers. Development Research Production Sale
Fact 3: Underutilization • Rule of thumb – server capacity should be twice what is required to handle peak time. Server Capacity 100% Resource wasted most of the time Peak Server Utilization 50% time
The Idea • Fact 1: Most applications are based on client-server model. • Fact 2: Business units have their own servers. • Fact 3: Server capacity should be twice what is required to handle peak time. • Application servers are underutilized most of the time. The idea - “Stealing” wasted machine cycles for backup applications.
Sharing Resource /w Applications Backup Application Server 2 Application Server 1 Virtual Backup Server Unused resources Application Server 3 Application Server 4 Backup
Virtualization - Grid Computing Grid Computing Application Server 2 Application Server 1 Backup Node 3 Backup Node 2 Virtual Backup Server Backup Grid Backup Node 1 Application Server 3 Application Server 4
Benefits • Any underutilized server can be selected for each backup job • Higher performance • Fault tolerant • Better resource utilization • A backup grid shares hardware with application servers • Less hardware cost • Less system administrative cost
Job Scheduler Catalog Schedules services for various job requests. Job Monitor Resource Monitor Service Sets up and keeps track of all jobs in the system System Architecture Job Pre-processing Backup Operation Model Manages metadata System Administration Console Backup Grid Monitors servers and network utilization Local Storage Local Storage Local Storage Local Storage
Prototype - Features • Single-Instance Storage: Save storage and communication costs. • Data Replication: Enhance data availability. • Data Redirection: Avoid job resubmission. • Intelligent Scheduling: • Prioritize backup jobs • Balance workload and storage usage among nodes
DIR Resource Monitor F1 F2 Job Scheduler Catalog 2 4 1 3 Job Monitor 6 5 4 Backup without SIS F2 F1 HD Backup Operation When to Schedule? Where to store? Storage Node1 Storage Node2 Storage Node3 Storage Node4
DIR Resource Monitor F1 F2 Job Scheduler Catalog 2 4 1 3 Job Monitor 6 5 4 Backup with SIS F2 F1 HD Backup Operation Storage Node1 Storage Node2 Storage Node3 Storage Node4
DIR Resource Monitor F1 F2 Job Scheduler Catalog 8 Job Monitor 10 7 11 5 9 Backup with Redirection & SIS To handle insufficient disk space and avoid job resubmission F2 F1 HD Backup Operation Storage Node1 Storage Node2 Storage Node3 Storage Node4
DIR Resource Monitor F1 F2 Job Scheduler Catalog 1 4 2 3 Job Monitor 5 4 Restore with SIS F2 F1 HD Backup Operation Storage Node1 Storage Node2 Storage Node3 Storage Node4
Server 1 Server 2 Job Monitor Backup Operation Module Job Scheduler Catalog Service Intel P4 3.0GHz, 2GB, 80GB Intel P4 3.2GHz, 1GB, 80GB Storage Node 2 Storage Node 3 Storage Node 1 Storage Node 4 Local Storage Manger Local Storage Manager Local Storage Manager Local Storage Manager Local Resource Monitor Local Resource Monitor Local Resource Monitor Local Resource Monitor Intel P4 1.7GHz, 256KB, 10GB Intel P4 1.7GHz, 256KB, 10GB Intel P4 1.7GHz, 256KB, 10GB Intel P4 1.7GHz, 256KB, 10GB System Setting
Result of Redirection Node 1 transfer first part of Job 3 to Node 3 Job 3 is combined at Node 3 Job 3 is transferred to Node 3 Job 3 continues to transfer Job 3 finished Node 1 is full Node 3 combines the data for Job 3 Job 3 is redirect to Node 3
Concluding Remarks • New service-oriented framework for storage backup • Scalable, robust, efficient and economical • Experiment confirms the prototype • Reduce storage and communication • Better fault tolerance and load balance 24