SAN Device Virtualization

1. SAN DeviceVirtualization Commands hidden in NX-OS 5.x as feature has been deprecated after NX-OS 4.x

2. About SAN Device Virtualization (SDV) • What is SAN Device Virtualization? Virtualization of Initiators/Targets • Target Migration = Migrating from one target (Primary Target) to another (Secondary Target) for one or more initiators (servers). • Server Migration = Migrating from one initiator to another for one or more targets (disk). • Main Focus of presentation on Target Migration, but applies to Server Migration also. • What is driving the need for Target Virtualization functionality? • Target failures: hardware, logical corruption • Data migration: technology refresh, workload balancing, storage consolidation • Virtualization Ready: for future needs

3. About SDV continued • What are the benefits of the feature compared to manual migration? • Reduce the amount of time for migration; hence reducing the downtime • Ease of management; Reduces possibility of human errors. • Easily scalable for larger number of initiators/targets.

4. Today's Deployment for Handling Target Failures • Designed for HA; Redundancy is the key. • Deploy two arrays: primary & secondary. • Use some type consistency technology such as EMC SRDF between primary and secondary to ensure that secondary is a mirrored copy of the production LUN. • When primary fails, manually bring secondary online. All I/O will now take place with the secondary. • The time required to `use’ secondary is the problem. Primary Target I/O - Normal ASYNC Replication SAN I/O - After Primary Failure Secondary Target

5. The Challenges of Handling Target Failures • The time required to make secondary target accessible can be quite long. This impacts the application availability. • For a typical production environment of 2 clustered hosts, and a database size of 2TB, this takes over 4 hours. • This down-time consists of two components: • - Zoning Changes • All the initiators now have to be re-zoned with the Secondary Target • - Re-configuring certain initiators • Since WWN and FCID of the secondary is different, some driver files have to be changed and server rebooted, which adds risk (Eg: HP-UX and AIX servers) • Clustering (multiple initiators) compounds the problem; Procedure has to be repeated for each server of the cluster. Up to a couple of hours Zoning Changes Changes to Initiators Secondary Array online Primary Array fails

6. The SDV Manager • SDV is a conditional service running on the MDS Switch Supervisor that can create virtualized devices. • SDV presents a Virtual (Proxy) target to initiator. • Initiators are presented with a Virtual Device (VD);the virtualized form of the real target. • VD is created by configuration and is identified by a name. • VD virtualizes a single real target at any point of time. • VD registered with Name Server with a pWWN and consumes an FCID outside of the real domain. • VD is ‘hosted’ by one of the switches in the fabric. • Administrators Zones VD with (real) initiators using regular Zoning Config. Initiator can be a VD also. • Initiators discover VD through NS queries. • The initiators always perform SCSI I/O operation to the virtual address of the VD. • At this point, either the initiator or target can be VD, but not both.

7. SDV Manager Contd.. • SDV performs FCID translations on the frames destined to VD. • In the forward direction, DID is rewritten to that of the real target. • In the reverse direction, SID is rewritten to that of the VD. • The VD is hosted by one of the switches using a virtual domain. • The VD has an FCID and pWWN/nWWN assigned by the switch. • VD exists as long as its real counterpart is online. It inherits the FC-4 type and features from its real counterpart.

8. SDV Solution: The Big Picture • Configure a Virtual Target (VD) on MDS. • Initiator is zoned with VD. • VD is linked to a real (Primary) Target. • For control traffic (PLOGI, PRLI etc.) • MDS provides proxy functions. That is, it receives these ELS frames and re-transmits it to the real Target by performing NAT in header and payload. • For data traffic • MDS provides NAT of header VD FCID in hardware • The Primary and Secondary Targets can be on different MDS switches. • This features can be enabled on MDS without requiring a change on servers and storage arrays. • The admin can perform the Target Migration from primary to secondary through a single step configuration. WWN1FCID1 Initiator WWN4 FCID4 Virtual Target WWN2 FCID2 WWN3 FCID3 Secondary Target Primary Target

9. Logical Zones • The {initiator, VD} is zoned. However the real target is not part • of any zone. Hence according to zoning the reverse path • frames from target to initiator would be dropped. • To overcome this restriction a logical zone is created with • {initiator, target} and Zone Server is notified. • The purpose of this logical zone is to make sure that w.r.t. the • targets it is zoned with the initiators, but w.r.t. the initiators • they are not zoned with the target. • The logical zone is not present physically in the active • ZoneSet. Hence presence of this zone not known to others. • Logical Zones known only to SDV switches. Hence solution is • limited to those devices that are directly connected to SDV • enabled switch.

10. Solution Step 1: Creating the VD i3pwwni3 i2pwwni2 i1pwwn i1 SDV Virtual Target vtpwwn vt1 VT • A VD defined by user configuration enumerating all • the targets (primary and secondary) • When a VD is created • SDV picks a unique pwwn for the VD. • Registers the VD name with its pwwn as an Device Alias. t1pwwn t1 t2pwwn t2 Primary Secondary

11. Solution Step 2: Zoning VD with Initiators i3pwwni3 i2pwwni2 i1pwwn i1 SDV vtpwwn vt1 Virtual Device • When VD name is zoned with initiators and activated … • SDV assigns FCID for VD. • SDV Registers VD with NS and sends SW-RSCN for VD once the primary target is online. • Inform ZS about presence of Logical Zone (dotted • in the diagram). t1pwwn t1 t2pwwn t2 Primary Secondary

12. Solution Step 3: Linking VD with target i3pwwni3 i2pwwni2 i1pwwn i1 SDV vtpwwn vt1 Virtual Device • When VD is linked with real target • ACL Rewrite entries are programmed to do the header NAT of data frames on access ports. • ACL Capture entries are programmed for payload NAT of Control frames (ELS) by the SUP. t1pwwn t1 t2pwwn t2 Primary Secondary

13. Hosting the VD • Hosting involves injecting the virtual entities (VD) into • the fabric. • Hosting done by one switch; But it cannot use its local domain • for the VD FCID because if this switch leaves the fabric OR the • local domain changes, FCID would have to change. • The hosting switch reserves a Virtual Domain used for • assigning to VD. If the hosting switch goes down another • switch could take over its role using the same virtual domain. • Virtual Domain reservation to be used for the VD is through • Domain Manager using the RDI mechanism. • The route for virtual domain is advertised by hosting switch • through FSPF. • Hosting consists of: FCID allocation, NS registration, • SW-RSCN generation, Injecting Zones and route advertisement.

14. Rewrite Entries & ELS Capture entries • Rewrite entries are programmed in ACL TCAM for • FC header fcid-rewrite of the data traffic. • For each <initiator,target> pair two rewrite entries are needed. One in the • forward direction and one in the reverse direction. • Each ACL Rewrite entry includes zoning, forwarding and rewrite • information and avoids using FIB lookup/adjacency. The rewrite entry • would overwrite the zoning entry. • ELS capture entries are programmed for FC payload fcid • rewrite of the control traffic. • All the ELS frames to VD are punted to Supervisor. • The SDV module running on Sup does payload fcid-rewrite depending on • the ELS type and forwards it on the egress interface. • Some of the ACC frames also need payload fcid-rewrite.

15. i1 vt1 i1 t1 t1 i1 vt1 i1 Frame translation with rewrite entries i1pwwn i1 ACL entries Frame values Rewrite values S_ID D_ID S_ID D_ID i1 vt1 i1 t1 i2 vt1 i2 t1 ………. t1 i1 vt1 i1 t1 i2 vt1 i2 S_ID D_ID t1pwwn t1

16. Header and Payload rewrites PLOGI trapped and punted to supervisor for payload FCID rewrite Supervisor forwards frame to egress line card for header FCID rewrite SCSI commands are forwarded to egress line card for FCID header rewrite

17. Configuration Device-Alias created with this name automatically Real target PWWN • Enable the feature (config)# sdv enable • Create a VD(config)# sdv virtual-device name disk vsan 1 (config-sdv-virt-dev)# pwwn 21:00:00:20:37:a9:d7:42 primary • Zone VD with desired devices (config)# zone name zone1 vsan 1 (config-zone)# member pwwn <initiator> (config-zone)# member pwwn <VD-pwwn assigned by MDS> • Activate ZoneSet (regular zoneset activation) (config)# zoneset activate name <zs> vsan <> • Link with current primary real target (config-sdv-virt-dev)# link pwwn 21:00:00:20:37:a9:d7:42 • (Use the same ‘link’ command to switch to secondary target if primary goes down) (config-sdv-virt-dev)# link pwwn 22:00:00:20:37:a9:d7:42 Real initiator PWWN PWWN assigned by MDS to represent target to host Different pwwn that exposes same luns

18. Domain, FCID, and PWWN used for VD switch# show sdv virtual-device name disk virtual-device name disk vsan 1 [ WWN:50:00:53:00:00:cd:c0:01 FCID:0x140654 Real-FCID:0x7c03e4 ] pwwn 21:00:00:20:37:a9:d7:42 primary FCID used to represent the real target PWWN assigned by MDS. Use this for zoning with real initiator RTP9-CAE-POD2-9509# sh fcdomain domain-list VSAN 1 Number of domains: 2 Domain ID WWN --------- ----------------------- 0x64(100) 20:01:00:05:30:00:49:1f [Local] [Principal] 0x14(20) 50:00:53:07:ff:f0:00:71 [Virtual (SDM)] Domain used to represent DID for the Virtual Devices hosted by this switch for this VSAN

19. Locking down the domain and FCID used for the VD • The hosting switch will reserve a DID for the VDs • It can not be the same DID assigned to the VSAN in the event that another switch has to assume the hosting of the VD. switch(config-sdv-virt-dev)# ? Configure a Virtual-device: device-alias Add a device-alias to the Virtual-device do EXEC command end Exit from configure mode exit Exit from this submode link Link the Virtual-device to a real device no Negate a command or set its defaults pwwn Add a pwwn to the Virtual-device virtual-domain Configure the persistent virtual domain virtual-fcid Configure the persistent virtual fcid Configured under virtual device sdv enable sdv virtual-device name disk1 vsan 1 virtual-domain 138 virtual-fcid 0x8a197f pwwn 21:00:00:04:cf:17:66:b7 primary sdv commit vsan 1 Completed Configuration

20. Multi-switch Fabric • SVD enabled on one of the switches that acts as the hosting switch. Eg: SW1 is the hosting switch • The frame rewrite has to happen only • on this hosting switch. Eg: Even though the traffic between I2 and T1 need not go thru SW1, it goes thru SW1 (because of VD’s fcid) • Drawback: Does not work if any switch • in path from initiator to target is not • SDV enabled. (logical zone unaware) • Other Drawbacks: • - The hosting switch is the single • point of failure. • - Hosting switch failure recovery • requires manual intervention. • - Non optimal routing of data frames. I2 I1 VD SW2 SW3 SW1 T1 T2 Primary Secondary

21. Comparison with IVR-NAT

22. Q and A