400 likes | 643 Views
Mass Storage & Information Retrieval. Paul J Mazzotte Union University. April 02, 2004. Agenda. Background RAID and JBOD SCSI and FC Storage Paradigms DAS (Direct Attached Storage) NAS (Networked Attached Storage) SAN (Storage Area Networks) Performance and Cost – NAS vs SAN
E N D
Mass Storage & Information Retrieval Paul J Mazzotte Union University April 02, 2004
Agenda • Background • RAID and JBOD • SCSI and FC • Storage Paradigms • DAS (Direct Attached Storage) • NAS (Networked Attached Storage) • SAN (Storage Area Networks) • Performance and Cost – NAS vs SAN • Storage and Backup • Backup Software • Tape Technologies • DAS and Backup • SAN and Backup • What’s Next
RAID and JBOD • JBOD: “Just a Bunch Of Disks” • Drives independently attached to the I/O channel • Scaleable, but requires server to manage multiple volumes • Does not provide protection in case of failure • RAID: “Redundant Array of Inexpensive Disks” • Fault-tolerant grouping of disks that server sees as a single volume • Combination of parity-checking, mirroring, and striping • Self-contained manageable unit of storage
RAID • Multiple RAID Levels to choose from: • 0, 1, 2, 3, 4, 5, 6, 10 • Each level has certain inherent advantages and disadvantages.
RAID Levels • Data is subdivided and each division is written to a different disk drive. • Advantages – Performance when multiple controllers used • Disadvantages - Not a true raid • Minimum 2 drives • Data is written to two different drives. • Advantages – 100% Redundant • 1 write, 2 reads possible • Disadvantages – Highest Disk Overhead • Minimum 2 drives
RAID Levels • The data block is subdivided ("striped") and written on the data disks. The stripe parity is generated on writes, recorded on the parity disk and checked on reads. • Advantage – Medium read, High write performance • Disadvantages - Rebuild time (Compared to Raid 1) • Minimum 3 drives • Each entire data block is written on a data disk; parity for blocks in the same rank is generated on Writes, recorded in a distributed location and checked on Reads. • Advantage – High read, medium write performance • Disadvantages – Rebuild time (Compared to Raid 1) • Minimum 3 drives
SCSI Version Databus Speed Cable 1 (1986)8 bit5 MB/s (slow) 6 meters 2 (1994) 8 bit (narrow) 10 MB/s (fast) 25 meters 16 bit (wide) 20 MB/s 25 meters 3 [Ultra](1995) 8 bit 20 MB/s (fast-20) 25 meters 16 bit 40 MB/s 25 meters [Ultra-2](1998) 8 bit 40 MB/s (fast-40) 25 meters 16 bit 80 MB/s 25 meters [Ultra-3](1999) 8 bit 80 MB/s (fast-80) 25 meters 16 bit 160 MB/s (ultra-160) 25 meters [Ultra-4](2003) 8 bit 160 MB/s (fast-160) 25 meters 16 bit 320 MB/s (ultra-320) 25 meters
200 MB 200 MB 200 MB 200 MB Fibre Channel Point-to-Point 200 MB Arbitrated Loop Switch Switched Fabric
SCSI and FC Fibre Fibre Parallel Channel Channel AL SCSI Connections 16 Million 126 15 Distance 10 km 10 km 25 m Bandwidth 200 MB/s 200 MB/s 320 MB/s Per connection Shared Shared Hut Plug Yes Yes No Multiple Protocols Yes Yes No
SCSI-3 IP ATM ULP (Upper Level Protocol) SCSI - 3 Command Set Mapping IPI - 3 Command Set Mapping (IPI-3 STD) FC Link EncapsulationFC - LE FC - ATM FC - 4 FC - 3 Common Services FC - 2 Framing Protocol Fibre Channel Physical & Signaling Interface( FC- PH, FC-PH2, FC-PH3 ) FC - AL FC - AL -2 FC - 1 Encode / Decode 8B/10B Encoding FC - 0 Copper, Optical Physical Variant NOT SCSI vs FC
Block I/O (SCSI/FC-AL) DAS Client Workstations LAN File I/O (NFS/CIFS) File Server(s) Application Server(s) Definition: DAS is composed of multiple storage disks or disk array units that are directly attached to a general purpose server.
DAS Issues • Proliferation of “server and storage islands” which causes a large management burden • File Sharing Issues
NAS Client Workstations LAN File I/O (NFS/CIFS) NAS Servers (filers) Definition: NAS is a special-purpose storage system that directly attaches to the LAN and responds to file I/O requests coming across the LAN from a device.
Same as DAS – Not Exactly • Tuned Network Operating System (NOS) • Supports Multiple Protocols (NFS, CIFS, NCP)
Does NAS Solve DAS Issues • Simplify Management – Yes (for the most part) • Allows storage to be consolidated but only up to the size of the NAS box (~5 to 15 TB) • File Sharing – Yes • “True NAS” servers will have support for multiple protocols.
NAS Issue • Performance • Network bandwidth / Network Traffic • Protocol Inefficiencies
SAN Client Management Station LAN Management Server(s) Application Servers Block I/O (FC) FC Network Definition: SAN is a high-speed network dedicated to interfacing storage subsystems to servers. Disk
FC Switch Network Zone Y Zone X Node Node Node Node Node Zoning (1 of 2) Zoning arranges FC connected devices into logical groups
Zoning (2 of 2) • Operation • Zone members “see” only other members of the zone • Zones are configured dynamically • Devices can be members of more than one zone • Switched fabric zoning can take place at the port or device level • Benefits • Secured device access • Allows operating system co-existence
Does SAN Solve DAS Issues • Simplify Management – Yes • Allows storage to be consolidated (seen as one big island instead of a couple large islands like NAS) • File Sharing – Not Yet • Still waiting for the development of a CFS.
SAN Local storage access Private net for storage Storage protocols Centralized management SAN and NAS Recap NAS • Remote file access • Shares user net • Network protocols • “Centralized” management Good for file sharing (“home directories”) Good for hosting large databases
SAN/NAS Performance SPEC
SAN/NAS Cost 3 Year TCO (cents per MB) for 2 TB Cost per MB “The Storage Report - Customer Perspectives & Industry Evolution - 19 June 2001” by Merrill Lynch & Co. and McKinsey & Company, Page 48, Chart 51
Netapp FAS960 SAN/NAS Cost Cents per MB (12 TB) Cents per MB (5 TB) Cents per MB (2.5 TB) Type Platform NAS 7.2 ($176,722) 4.1 ($206,836) N/A 3.4 ($406,880) 5.5 ($275,266) Compaq EVA 9.1 ($228,261) SAN Note: SAN costs include two 16-port switches but no cabling.
SAN/NAS Business Trend “SNIA Presentation - 19 May 1999” by Nick Allen of Gartner Group
DAS 90% SAN 80% 16 14 SAN % 70% 12 60% 10 50% 8 40% 6 30% 4 20% 2 0 10% 0% 2001 2002 2003 2004 2005 2006 2000 SAN/NAS Business Trend Annual vendor revenue $B Source: “Worldwide external raid controller-based storage forecast, 2000-2006”, Gartner, August 2002
Backup Software (Mid-Range) • Legato (Networker) • Veritas (Netbackup) • IBM (Tivoli)
Mid-Range Tape Technologies AIT-3 SuperDLT LTO-1 Mammoth-2 Manufacturer Sony Quantum IBM/S/HP Exabyte Release Q4 2001 Q1 2001 Q3 2000 Q1 2000 Technology Helical Linear Linear Helical Native Capacity (GB) 100 110 100 60 Compressed Capacity (GB) 260 220 200 150 Native Transfer Rate(MB/s) 12 11 15 12 Compress Transfer Rate (MB/s) 31 22 30 30 12 Hr Window Trans Rate (GB) 518.4 475.2 648.0 518.4 MTBF (Hours) 400,000 250,000 250,000 300,000 Head Life (Hours) 50,000 30,000 30,000 50,000 Media Life (Avg Passes) 30,000 1,000,000 1,000,000 20,000 Media Price per Cartridge $135 $134 $110 $89 Price per GB (Native) $1.35 $1.22 $1.10 $1.48 Drive Price $?,?00 $4,400 $4,300 $4,000 SCSI LVD LVD/HVD LVD/HVD LVD/HVD Fibre Channel NO NO YES YES The announced road maps are as follows: [Note: Year(Native Capacity, Compressed Capacity, Native Transfer Rate, Compressed Transfer Rate] Mammoth (M3, M4, M5)2003(120,300,20,50) 2004(200,500,30,75) 2005(400,1000,60,150) LTO (LTO-2, LTO-3, LTO-4)2003(200,400,30,60) 2004(400,800,60,120) 2006(800,1600,120,240) AIT (AIT-4, AIT-5, AIT-6) 2003(200,520,24,62) 2005(400,1040,48,124) 2007(800,2080,96,248) DLT (SDLT-2, SDLT-3) 2003(220,440,22,44) 2005(500,1000,44,88) 200?(???,????,??,???)
Backup Servers Small Servers / Desktops Backup Client Nodes Jukebox Jukebox More Servers DAS and Backup LAN
Files to Backup Backup File Index Disk Blocks Servers (Oracle, Mail, etc) FC Network SAN and Backup LAN Backup Server NAS Nodes Server Nodes From Gigabit Netapp Filers SAN Disk Array(s) Tape Library
In The Near Future • Storage • iSCSI • Backup • Disk to Disk Backup
Review • RAID and JBOD • SCSI and FC • NAS and SAN • Backup