Beyond NAS and SAN: The Evolution of Storage

1. Beyond NAS and SAN:The Evolution of Storage Marc Farley Author Building Storage Networks

2. What We’ll Do • Analyze storage network functions • Apply these functions in NAS and SANs • Observe market pressures forcing changes • Identify evolving architectures

3. Building Blocks Fundamental Functions of Storage Networks • Wiring • Fibre Channel. Parallel SCSI, Ethernet, IP • Storing • Devices & Subsystems, I/O Controllers, Virtualization SW & HW (RAID, Volume Managers, Mirroring) • Filing • File systems, Databases, Backup, Replication

4. Building Blocks Traditional SAN Functional Distribution Storing Controller HBA Filing Wiring StorageSubsystem Fibre Channel Network Host System

5. Building Blocks Traditional NAS Functional Distribution InternalBus NIC Storing Filing Filing Filing Wiring LAN Client System File ServerSystem

6. Building Blocks Wiring: Physical & Logical Parts • Physical = Cabling and Network Hardware • signaling, encoding, error detection • Logical = Algorithms, Addressing, Protocols and Services • access methods, flow control (buffer management) • addressing, naming, topology management, routing • filtering, zoning • gateway, bridging and tunneling services

7. Building Blocks Wiring: Fibre Channel and i-SCSI • Functionally equivalent as wiring technologies • More of a business issue than a technology issue • Question: Will i-SCSI SANs be implemented as distinct networks or grow as part of the existing Ethernet/IP infrastructure? • i-SCSI leverage might be less than expected - but it might be good enough

8. Good Idea Wiring: Separate SANs or Bigger LANs? LAN SAN LAN

9. An Idea An Question for the Audience….. • What is the best routing method for storage traffic? • To date, storage networks have borrowed from IP networking • OSPF in IP networks (NAS) & FSPF in Fibre Channel • What do you want? • Fast recovery & optimal availability • OR • Compatibility with IP routing

10. Building Blocks Storing • Block level operations • storage block addressing • store, retrieve, status, resource reservation • Subsystems and Virtual Devices • block translation, mirroring, multiple ports, LUN masking • I/O commands between initiators and targets • SCSI, serial SCSI, SCSI adaptations, IDE • Error correction really happens here

11. Storing on Steroids: Virtualization • Virtualization techniques expand and extend the capabilities of devices and subsystems • Virtualization as a storing level function has no ability to manage anything about its contents Physical Devices VirtualStorage

12. Building Blocks Filing • Representation of data • files, directories, tables, records • Storage data structure (block address layout) • equalized performance, maintain free blocks • System kernel integration • kernel manages time, file system manages space • File semantics • Open, close, update, delete, block range operations

13. Building Blocks The Fundamental Role of Filing • Manage the use of the storing address space Storing Filing

14. Internal Functions in Filing

15. NAS & SAN REDUX

16. Gotcha SANs are the Application of Storing Functions Over a Network • The Caveat: SANS do not convey any knowledge of data structures • & there is no way to provide data-level management Storing Storing Storing Functions Initiators Network Targets

17. But not data structures Virtualization Creates Storing Structures Physical Devices Storing Storing Block Data Initiators Network Virtual Targets

18. Gotcha NAS: The Application of Filing Functions over a Network • The Caveat: ‘Loose wiring’ for I/O intensive computing Filing Filing Filing Functions Servers Clients Network

19. Challenge for NAS: Distribute Filing Functions & Reinforce the Wiring File Data Servers Clients Network

20. Storage Network Requirements

21. Best Practice Market Needs • Integrity • Recoverability • Security • Availability • Manageability (Scalability)

22. Gotcha Data Integrity Must be Maintained • Error free transmissions • Fiber optic cabling • Data segregation is a good thing • Who wants a Battle Royale over data? • Any to any connectivity is not required • Storage networks differ a lot from data networks • Zoning, masking and fencing are band-aids for existing architectural weaknesses • Filing enforces data integrity through locking

23. Gotcha Data Recoverability = Redundancy • Data redundancy  system redundancy • Mirroring is fast, but relatively inefficient • Extended distance mirrors are “open territory” • System buffer management is a caveat • Data structures must have integrity • Journalled file systems, database commitments • File system intelligence is most efficient • Serverless backup, data snapshots, replication

24. Gotcha Security - the Great Afterthought • Protecting Data From Theft • File systems have done this for decades • System-login is the gatekeeper • There are no constructs for storing-level security • Wiring-level security can be done, but is slow • Encryption is S-L-O-W for I/O • IP-SEC may provide a method

25. Remember the Good Old Days of SCSI? • Data integrity was less exposed with Parallel SCSI • Human error was far less likely • Storage was not accessible to hackers • Processing power on the storage end is needed to provide security functions

26. Data Availability • Remove the system as a bottleneck • (And integrity and security become more suspect) • Availability doesn’t necessarily mean “seamless” • Single servers in SANs can failover using redundant paths • NAS network failures must re-connect • Data sharing can provide availability • NAS file locking can ensure integrity for server farms • Data sharing in SANs is expensive and complex

27. Secret The Fine Print in Scalability • Maxed-out NAS requires additional mount points • Users and application configurations must change • SANs can use virtualization • Does “V” ensure integrity? • Is “V” recoverable? (will it support serverless backup?) • File system data structures must be included for non-stop scalability!

28. The Fine Print in Scalability • How does the filing functionknow about the newstoring address space? Storing ? ? ? Storing Filing

29. Evolving Architectures

30. Good Idea Evolving Storage Network Architectures • Filing technologies are intimately involved with: • Integrity • Recoverability • Security • Availability (sharing) • Scalability • Performance (efficiency - working smart) • Hmmmmm? Maybe filing could be important!

31. DAFS • Network Appliance (and others?) • Puts rigor into NAS wiring • Important proof of concept for NAS • Eliminate doubts about NAS for database I/O

32. Relocating Filing Functions in the Storage Network • Systems need to have a data view • Can be achieved by aggregating views from distributed • Metadata is moved out of host systems • Distributed or network-central • Data structures can be managed by the subsystem • Enables subsystem-based recoverability • Delivers enormous scalability

33. Aggregating Data Views • Virtualization at the file level Data views from many subsystems are merged

34. Independence of Metadata from Hosts • Integrity (locking) • Scalability Systems 1 2 3 Metadata Storage

35. Independence of Metadata from Hosts Systems Distributed Metadata Storage

36. Subsystem-Managed Data Structures System without data structure function Data structure is distributed across multiple subsystems