320 likes | 468 Views
EDC CR1 Storage Architecture. August 2003 Ken Gacke Systems Engineer (605) 594-6846 Gacke@usgs.gov. Storage Architecture Decisions. Evaluated and recommended through engineering white papers and weighted decision matrices Requirements Factors Reliability – Data Preservation
E N D
EDC CR1 Storage Architecture August 2003Ken Gacke Systems Engineer (605) 594-6846 Gacke@usgs.gov
Storage Architecture Decisions • Evaluated and recommended through engineering white papers and weighted decision matrices • Requirements Factors • Reliability – Data Preservation • Performance – Data Access • Cost – $/GB, Engineering Support, O&M • Scalability – Data Growth, Multi-mission, etc. • Compatibility with current Architecture • Program/Project selects best solution
Storage Technologies • Online Storage Characteristics • Immediate Data Access • Server Limitations • Number of I/O slots • System Bandwidth • Cost is Linear • High Performance RAID -- $30/GB using 146GB drives • Low Cost RAID -- $5/GB using ATA or IDE Drives • Non RAID – Less than $5/GB using 146GB drives • Facility Costs • Disk drives are always powered up • Increased cooling requirements • Life cycle of 3 to 4 years
Storage Technologies • Online Storage • Direct Attach Storage (DAS) • Storage directly attached to server • Network Attach Storage (NAS) • TCP/IP access to storage typically with CIFS and NFS access • Storage Area Network (SAN) • Dedicated high speed network connecting storage devices • Storage devices disassociated from server
Storage Technologies • Direct Attach Online Storage • Disk is direct attached to single server • System Configuration • SCSI or Fibre Channel • RAID Fibre Channel devices are typically SAN ready • Just a Bunch of Disk (JBOD) • Redundant Array Independent Disk (RAID) • High Performance on the local server • Manageability • Simple Configuration • Resource reallocation requires physical move of controllers and disk
Storage Technologies • Direct Attach Online Storage • Advantages • High performance on local server • Good for image processing and database applications • Disadvantages • Data sharing limited to slower network performance • Difficult to reallocate resources to other servers
Host A Host C File System File System File System Storage Technologies Direct Attached 100Mb Network (FTP/NFS) Host B 100MB FC
Storage Technologies • NAS Online Storage • Disk attached on server accessible over TCP/IP Network • System Configuration • Fibre Channel RAID Configurations • Switched Network Environment • Performance • Network Switches and/or dedicated network topologies • Reliability • NAS Server performs a single function thereby reducing faults • RAID, Mirror, Snapshot capabilities • Easy to Manage
Storage Technologies • Network Attach Online Storage • Advantages • Easy to share files among servers • Network Storage support NFS and CIFS • Servers can use existing network infrastructure • Good for small file sharing such as office automation • Availability of fault protection such as snapshot and mirroring • Disadvantages • Slower performance due to TCP/IP overhead • Increases network load • Backup/Restore to tape may be difficult and/or slow • Does not integrate with nearline storage
Host A Host B Host C File System File System File System Share Files Storage Technologies Network Attached 1Gb Network (NFS/CIFS) NAS Server
Storage Technologies • SAN Online Storage • Disk attached within Fabric Network • System Configuration • Fibre Channel • RAID Configurations • Scalable High Performance • High Reliability with redundant paths • Manageability • Configuration becomes more complex • Logical reallocation of resources
Storage Technologies Redundancy SAN Configuration Fibre Switch 100Mb Network Host A Host B Fibre Switch Host C (DMF)
Storage Technologies • SAN Online Storage Architecture • Disk Farm • Multiple servers share large disk farm • Server mounts unique file systems • Clustered File Systems • Multiple servers share a single file system • Software Required – Vendor solutions include • SGI CXFS • ADIC StorNext File System • Tivoli SANErgy
Logical reallocation of disk Storage Technologies Disk Farm SAN Configuration 100Mb Network Host A Fibre Switch Host B Host C
Clustered File System CXFS CXFS Storage Technologies Cluster SAN Configuration 100Mb Network Host A Fibre Switch Host B Host C
Storage Technologies • SAN Risks • Cost is higher than DAS/NAS • Technology Maturity • Solutions are typically vendor specific • Application software dependencies • Infrastructure Support • Complexity of Architecture • Management of SAN Resources • Sharing of storage resources across multiple Programs/Projects
Storage Technologies • SAN Benefits • Administration flexibility • Logically move disk space among servers • Large capacity drives can be sliced into smaller file systems • Scales better than direct attach • Integrate within nearline configuration • Data Reliability • Storage disassociated from the server • Fault Tolerant with Redundant Paths • Increase Resource Utilization • Reduce the number of FTP network transfers • Logically allocate space among servers
Clustered File System Tape Library Storage Technologies SAN with Nearline Configuration Fibre Switch 1Gb Network Host A Host B CXFS Host C DMF/CXFS
Online/Nearline Cost Comparison Use of Existing Infrastructure (CR1 Silo)
Storage Technologies • Bulk RAID Storage Considerations • Manageability • Server connectivity constraints • Many “islands” of storage • Multiple storage management utilities • Multiple vendor maintenance contracts • Data Reliability • Loss of online file system requires full restore from backup • On average, could restore one to two terabyte per day • Performance • Multiple user access will reduce performance • Life Cycle • Disk storage life cycle shorter then tape technologies
Storage Technologies • SAN Nearline Storage • Data Access • Data stored on infinite file system • Immediate access to data residing on disk cache • Delayed access for data retrieved from tape • Access via LAN using FTP/NFS • Access via SAN Clustered File System • SGI DMF/CXFS Server • SGI, SUN, Linux, NT clients
Storage Technologies • SAN Cluster Proposal • Mass Storage System & Product Distribution System (PDS) • Limit Exposure to Risk • Servers are homogeneous • Implement with Single dataset • Data is file orientated • Data currently being FTP • Anticipated Benefits • Improved performance • Reduce total disk capacity requirements • Experience for future storage solutions
Current DMF/SAN Configuration Product Distribution CXFS SAN Storage DMF Server 1Gb Fibre 2Gb Fibre Disk Cache /dmf/edc 68GB /dmf/doqq 547GB /dmf/guo 50GB /dmf/pds 223GB /dmf/pdsc 547GB Tape Drives 8x9840 2x9940
CR1 Mass Storage Av 12.1MB/sec Description 1996 – 3490, pre DOQQ 1999 – D-3, DOQQ 2002 – 9840, DOQQ 2003 – 9840/9940, UA/AVHRR
Storage Technologies • Multi Tiered Storage Vision • Online • Supported Configurations • DAS – Local processing such as image processing • NAS – Data sharing such as office automation • SAN – Production processing such as product generation • Data accessed frequently • Nearline • Integrated within SAN • Scalable for large datasets and less frequently accessed data • Multiple Copies and/or Offsite Storage
Storage Technologies • SAN – Final Thoughts • SAN Technology Maturity • SAN solution should be from a single vendor • Program/Project SAN solution benefits • Decrease storage requirements • Increase performance • Increase reliability • Increase flexibility of resource allocations • Increase cost (hardware/software) • Increase configuration complexity