1 / 40

Cluster Storage & Scale-Out File Server: Overview and Options

Learn about cluster storage options, shared storage requirements, and the advantages of using Cluster Shared Volumes (CSV) and SMB File Shares. Explore Software-Defined Storage and the benefits of Software-Defined Storage in Windows Server.

laurela
Download Presentation

Cluster Storage & Scale-Out File Server: Overview and Options

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 04 | Cluster Storage & Scale-Out File Server Elden Christensen | Principal Program Manager Lead | Microsoft Symon Perriman | Vice President | 5nine Software

  2. Module Overview • Introduction to Cluster Storage • Software-Defined Storage • SMB Storage • Scale-Out File Server

  3. Introduction to Cluster Storage

  4. Cluster Storage OptionsFlexible storage choices for building clusters Shared Storage Data Replication Application Replication Software Replication SAS RBOD FC Hardware Replication SMB Spaces RAID HBA SAS JBOD Example: Exchange SQL AlwaysOn 3rd party software replication solution iSCSI FCoE

  5. Shared Storage Requirements • Basic disk • Dynamic Disks require 3rd party add-on • GPT or MBR partition type • NTFS or ReFS file system • SCSI-3 Commands • Persistent Reservations (PRs) required • Multipath IO (MPIO) recommended • Storage Spaces (Pools) supported

  6. Cluster Disk Types – Traditional Cluster Disk • Storage Area Network (SAN) disk • Supported by all workloads • Accessed by one node at a time • Accessed using a drive letter or volume path • When unused, listed under Available Storage • Quorum/witness disk

  7. Cluster Disk Types - Cluster Shared Volume (CSV) Disk • Clustered File System • Storage Area Network (SAN) disk • Supported by Hyper-V, Scale-Out File Server & SQL only • Accessed by multiple nodes at a time • Accessed using C:\ClusterStorage\Volume# • Access is managed by the “Coordinator Node”

  8. CSV File System Single namespace Aggregate all volumes No more drive letters Leverages mount points Fault tolerance Node, network and storage failures Zero downtime CHKDSK Fast failover No drive ownership changes Performance Scalability Availability Operability Shared access Enables all nodes to access a common volume Simple management Manage from any node Abstraction to which node owns the disk CSV Block Cache 7x faster VDI VM boot time (avg.) Block level I/O redirection

  9. CSVFS Cluster-Wide File System Cluster-wide namespace with fast failure recovery CSV File System Proxy file system on top of NTFS/REFS Mounted on every node Decides direct IO vs. file system redirect IO CSV Volume Manager Responsible for the creation CSV volumes Direct IO for locally attached spaces and Block-level IO redirect for non-locally attached spaces CSV Filter Attaches to NTFS/REFS for local clustered spaces Controls access to the local NTFS file system Co-ordinates metadata operations CSVFS supports most NTFS features and operations Diff VHDX, 8GB cache, 5,120 virtual machines (16 host), From VM state change to user logon complete

  10. Cluster Storage Types – SMB File Share • Windows Server (SMB 3) File Share • Supported by Hyper-V only to store virtual hard disks • Accessed by multiple nodes at a time • Accessed by \\<sharename>\<diskname>.vhdx • Recommended to deploy with a Scale-Out File Server (SOFS)

  11. iSCSI Target Server SCVMM VHDX Support • Feature included in Windows Server • Presents VHD’s as iSCSI LUN’s on the network • Provision LARGER (up to 64TB) LUs • Online expand/shrink of LUs • Provision dynamically-growing LUs SMI-S SMI-S Support • Fully manage iSCSI Target Server using SMI-S • Empowers full end-to-end management with System Center Virtual Machine Manager • Standards-based management iSCSI Target Server

  12. Guest Clustering with Shared Virtual Disks Guest Clustering Guest Clustering with commodity storage Sharing VHDX files provides shared storage for Hyper-V Failover Clustering Maintains separation between infrastructure and tenants • Scale-Out File Server for file based storage • Cluster Shared Volumes (CSV) on block storage Block Storage File Based Storage Virtual SAS VM presented a shared virtual SAS disk Appears as shared SAS disk to VM VHDX VHDX

  13. Manage Cluster Disks

  14. Software-Defined Storage

  15. New Windows Server Approach to Storage Strategic shifts to reducing storage costs with Windows Server 2012 Software-Defined Storage File Based Storage High Performance SMB Protocol for Hyper-V Storage over Ethernet networks Storage Spaces Cost-Effective Business Critical Storage

  16. Infrastructure-as-a-Service Storage Vision • Dramatically lowering the costs and effort of delivering IaaS storage services • Disaggregated compute and storage • Independent manage and scale at each layer • Industry standard servers, networking and storage • Inexpensive networks • Inexpensive shared JBOD storage System Center Hyper-V Clusters SMB Scale-Out File Server Shared JBODStorage

  17. Deconstructing a SAN Physical Disks Controllers Connectivity Adaptors The brains of the SAN – typically now with x86 CPU, Memory, and provides enterprise features like Thin Provisioning, Deduplication, Storage Tiering etc. Multiple controllers provide resiliency. Resilient connectivity to external sources viaiSCSI, FC, FCoE, NFS, SMB Flash-based (SSD) or spinning media (HDD) to provide the raw storage capacity for your data. Pooled by the controllers,and sliced into LUNs (Simple, Mirrored, Parity etc.)

  18. Microsoft Storage Management Physical Disks Microsoft is now the controller Connectivity Adaptors Clustered Windows Server 2012 R2 File Servers (SOFS)creates disk pools, then slices them into Storage Spaces.Spaces can be Thin Provisioned & support Deduplication.Spaces can be Simple, Mirrored or Parity. Windows Server File Servers have resilient connectivity to external sources using regular 1GbE, 10GbE Network Adaptors. Support for up to 56Gb RDMA Adaptors. Support via iSCSI, SMB 3.0 & NFS Connectivity Low cost, low complexity JBOD shelf with SSD/HDD mixand multiple SAS connectivity ports

  19. SCVMM Storage Management Pillars Insight Flexibility Automation End to End Discovery and Mapping Block and File Provisioning Rapid Provisioning Copy File, Diff Disks Pool, Volume, and File Share Classification Scalable Provisioning Scale-out File Server Bare Metal Provisioning Storage Monitoring and Capacity Trending Extensive Device Support VM, Host, and Cluster Storage Management Standards Based Management Allocation and Assignment Disaster Recovery

  20. Standardizing Storage Management for Windows • Windows Server now has a single standardized management interface to manage storage • Storage Management API (SM-API) • SCVMM manages Storage Spaces and Scale-Out File Server through SM-API System Center Virtual Machine Manager Windows Server 2012 Server Manager ISV or Storage Vendor Applications Storage Management API (WMI) CIM Pass Through SMI-S compliant Fibre Channel switch SMP based subsystem SMI-S compliant subsystem Storage Spaces compatible JBOD SMI-S compliant NAS Storage Spaces

  21. End-to-End Storage Management Capacity Management Pool/volume/file share classification File share ACL management VM workload deployment to file shares Hyper-V Clusters Bare metal deployment of file server Creation of scale-out file server cluster Adding/removing file server nodes File share management Scale-out File Server Deployment SMB Scale-Out File Server Clusters Storage Spaces Virtualization and Resiliency Spaces Provisioning Discovery of physical spindles Storage pool creation and deletion Mirror and Parity Spaces creation and deletion Shared JBODStorage

  22. Storage Spaces – Storage Tiering • Optimized Data Placement • Pool consists of both SSDs and HDDs • Tiers within a Storage Space • Hot Data (SSDs), Cold Data (HDDs) • Sub-File-Level Data Movement • Complements write-back caching • Administrative Controlled Pinning • Pin hot files to faster SSD tier • Example: VDI Pooled VM’s VHD Virtual Machines Storage Space SSD Tier 400GB EMLC SAS SSD Cold Data Hot Data HDD Tier 4TB 7200RPM SAS HDD

  23. SMB Storage

  24. Hyper-V over SMB What is it? • Store Hyper-V files in shares over the SMB 3.0 protocol(including VM configuration, VHD files, snapshots) • Works with both standalone and clustered servers (file storage used as cluster shared storage) Highlights • Increases flexibility • Eases provisioning, management and migration • Leverages converged network • Reduces capital and operational expenses Supporting Features • SMB Transparent Failover - Continuous availability • SMB Scale-Out – Active/Active file server clusters • SMB Direct (SMB over RDMA) - Low latency, low CPU use • SMB Multichannel – Network throughput and failover • SMB Encryption - Security • VSS for SMB File Shares - Backup and restore • SMB PowerShell and VMM Support - Manageability Hyper-V Cluster Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V Hyper-V SQLServer SQLServer SQLServer IIS IIS IIS VDIDesktop VDIDesktop VDIDesktop File Server Cluster File Server File Server Shared Storage

  25. SMB Transparent Failover • Failover transparent to server application • Zero downtime – small IO delay during failover • Supports planned and unplanned failovers • Hardware/Software Maintenance • Hardware/Software Failures • Load Rebalancing • Resilient for both file and directory operations • Requires: • File Servers configured as Windows Failover Cluster • Windows Server 2012 on both the servers running the application and file server cluster nodes • Shares enabled for “continuous availability” (default configuration for clustered file shares) • Works for both classic file server clusters (cluster disks) and scale-out file server clusters (CSV) 1 Normal operation Failover share - connections and handles lost, temporary stall of IO 2 Connections and handles auto-recovered Application IO continues with no errors 3 Hyper-V 1 3 \\fs\share \\fs\share 2 File Server Cluster File Server Node A File Server Node B

  26. SMB Multichannel Full Throughput • Bandwidth aggregation with multiple NICs • Multiple CPUs cores engaged when NIC offers Receive Side Scaling (RSS) Automatic Failover • SMB Multichannel implements end-to-end failure detection • Leverages NIC teaming (LBFO) if present, but does not require it Automatic Configuration • SMB detects and uses multiple paths Sample Configurations Single 10GbE RSS-capable NIC Multiple 1GbE NICs Multiple 10GbE in LBFO team Multiple RDMA NICs SMB Client SMB Client SMB Client SMB Client LBFO NIC 10GbE/IB NIC 10GbE/IB NIC 10GbE NIC 10GbE NIC 10GbE NIC 1GbE NIC 1GbE Switch 10GbE/IB Switch 10GbE/IB Switch 10GbE Switch 10GbE Switch 1GbE Switch 1GbE Switch 10GbE SMB Server SMB Server SMB Server SMB Server NIC 10GbE/IB NIC 10GbE NIC 10GbE/IB NIC 10GbE NIC 10GbE NIC 1GbE NIC 1GbE LBFO Vertical lines are logical channels, not cables

  27. SMB Direct (SMB over RDMA) Advantages • Scalable, fast and efficient storage access • High throughput with low latency • Minimal CPU utilization for I/O processing • Load balancing, automatic failover and bandwidth aggregation via SMB Multichannel Scenarios • High performance remote file access for application servers like Hyper-V, SQL Server, IIS and HPC • Used by File Server and Clustered Shared Volumes (CSV) for storage communications within a cluster Required hardware • RDMA-capable network interface (R-NIC) • Three types: iWARP, RoCE and InfiniBand • RDMA NICs should not be teamed (use SMB Multichannel) SMB Client SMB Server Application User Kernel SMB Client SMB Server NTFSSCSI Network w/RDMA support Network w/RDMA support Disk R-NIC R-NIC

  28. Hyper-V Live Migration over SMB • SMB as a transport for Live Migration of VMs • Delivers the power of SMB to provide: • RDMA (SMB Direct) • Streaming over multiple NICs (SMB Multichannel) • Provides highest bandwidth and lowest latency TCP/IP Compression SMB w/RDMA(no compression) Live Migration can take advantage of high speed networking Live migration can stream over multiple networks for improved bandwidth RDMA enables offloading CPU resources to NIC during live migration

  29. SMB Bandwidth Management File Server for library Storage Default Limit = 100 MB/s Common Infrastructure SMB being leveraged for VMs to access storage, distribution from VM library, and live migration Desire to manage bandwidth of different types of SMB communication Live Migration Limit = 500 MB/s Hyper-V host 1 Hyper-V host 2 Control Configurable SMB bandwidth limits per category Three defined SMB categories: Default, VirtualMachine and LiveMigration Storage No Limit Scale-out File Server VHDX

  30. Hyper-V Online Resizing and Storage QoS VHDX Resize Expand or shrink a VHDX based virtual disk with no downtime Hyper-V Storage QoS Maximum IOPS Setting to limit the maximum IOPS allowed to a virtual disk Minimum IOPS Alerts Notifications when specified minimum IOPS are not met for a virtual disk QoS VHDX VM Storage Metrics Storage attributes added to VM Metrics Average normalized IOPS Average latency

  31. Data Deduplication Hyper-V VDI Live VHD’s Deduplication of open VHD/VHDX files Gain space savings as high as 90% on VDI deployments with minimal impact Performance Faster read/write of optimized files Improved optimization speed SMB Direct Scale-out File Server Compatibility Support for Scale-out File Server with Cluster Shared Volumes (CSV) Dedup VHD

  32. Important notes on Hyper-V over SMB • Hyper-V supports SMB version 3.0 only • That includes Windows Server 2012 (SMB 3.00) and Windows Server 2012 R2 (SMB 3.02) • Thera are also 3rd-party SMB 3.0 solutions from storage partners like EMC and NetApp • The Hyper-V Best Practices Analyzer (BPA) will check the version of SMB • Active Directory is required • Computer accounts, which are required for configuring proper permissions, only exist in a domain • Continuously Available shares are recommended • Both Virtual Machine Manager 2012 SP1 and Virtual Machine Manager 2012 R2 support Hyper-V over SMB • Loopback configurations are not supported • File Server and Hyper-V must be separate servers • If using Failover Clusters, File Server and Hyper-V must be on separate clusters • Remote Management • Use PowerShell • Use Server Manager (for file shares) • Use Remote Desktop (RDP) • Use VMM 2012 SP1 • If using Hyper-V Manager remotely, Constrained Delegation is required

  33. Scale-Out File Server

  34. SMB Scale-Out • Targeted for server app storage • Example: Hyper-V and SQL Server • Increase available bandwidth by adding nodes • Leverages Cluster Shared Volumes (CSV) • Key capabilities: • Active/Active file shares • Fault tolerance with zero downtime • Fast failure recovery • CHKDSK with zero downtime • Support for app consistent snapshots • Support for RDMA enabled networks • Optimization for server apps • Simple management Hyper-V Cluster (Up to 64 nodes) Datacenter Network (Ethernet, InfiniBand or combination) Single Logical File Server (\\FS\Share) Single File System Namespace Cluster Shared Volumes File Server Cluster (Up to 8 nodes)

  35. SMB Scale-Out Capabilities Distributed Functionality Active-Active SMB shares accessible through all nodes simultaneously Distributed NetName (DNN) manages a DNS name fronting all physical node IPs Client round-robins through IPs (multiple parallel connects) Clients re-directed to “optimal” server node (CSV/Storage space owner) Management / Backup Extensive PowerShell Fan out requests where necessary Remote VSS (MS-FSRVP) Hyper-V Host Kernel User VM VM VMBUS VHD Parser B/W Limiter DPM SMB Client VSS SMBD WSK Witness Client RVSS Provider TCP NDK LBFO To Witness Service On SOFS node 2+ NIC Pair N NIC Pair 1 Witness Service RVSS Service SMB Server CSVFS DNN Resource Cluster Server Service Resume Key SOFS Resource VSS Shared VHD DB CSV Provider NTFS/ReFS LUN / Space Kernel User SOFS Node 1

  36. Interop Scale-Out File Server is for Hyper-V and SQL Server, not Information Worker!! 1 Only works if CA is enabled on shares 2 Not recommended on Scale-Out File Servers. 3 Not recommended on general use file servers. 4 Requires NTFS 5 CSC is less compatible with CA shares than the other IW technologies, due to how it decides a share is offline combined with the SMB 3 client. This means that Offline Files will stay online even if the user no longer has access to the share, for 3-6 minutes.

  37. Create a Scale-Out File Server

  38. Reviewing the Software Defined Storage Stack SMB Scale-Out File Server Access point for Hyper-V Scale-out data access Data access resiliency Cluster Shared Volumes Single consistent namespace Fast failover Storage Spaces Storage pooling Virtual disks Data Resiliency Hardware - Standard volume hardware - Fast and efficient networking - Shared storage enclosures SAS SSD SAS HDD Scale-Out File Server \\FileServer\Share Cluster Shared Volumes C:\ClusterStorage Software Defined Storage System Storage Space Virtual Disks Storage Pool Storage Node Storage Node Storage Node Storage Node Shared JBOD Storage

  39. Tips and Answers (from the blog) • Switch to the High Performance power profile • Make sure your network interfaces are RSS-capable • Use multiple subnets when deploying SMB Multichannel in a cluster • Disable 8.3 Naming (and strip those short names too) • Continuous Availability does not work with volumes using 8.3 naming or NTFS compression • Enable CSV Caching on Scale-Out File Server Clusters • Avoid loopback configurations for Hyper-V over SMB • Run the File Services Best Practices Analyzer (BPA) • Use PowerShell to find the free space on the volume behind an SMB file share • New per-share SMB client performance counters provide great insight • Minimum version of Mellanox firmware required for running SMB Direct in Windows Server 2012 • How much traffic needs to pass between the SMB Client and Server before Multichannel actually starts? • Is it possible to run SMB Direct from within a VM? • Can I use SMB3 storage without RDMA? • I only have two NICs on my Hyper-V host. Should I team them or not? • How to rebalance a Scale-Out File Server using a little PowerShell

More Related