380 likes | 714 Views
Migrating from Switch Clustering to Avaya Fabric Connect. Jeff Cox Senior Solutions Architect Avaya. @jeffcox65. Avaya VENA Campus Network Architecture. Avaya ERS 8800. Avaya VSP 9000. Campus Core/Distribution Virtualization using Shortest Path Bridging (SPB). Data Center 1 Core.
E N D
Migrating from Switch Clustering to Avaya Fabric Connect Jeff Cox Senior Solutions Architect Avaya @jeffcox65
Avaya VENA Campus Network Architecture Avaya ERS 8800 Avaya VSP 9000 Campus Core/Distribution Virtualization using Shortest Path Bridging (SPB) Data Center 1 Core Server Access Campus Core Distribution (optional) • Lossless v3.2 • SPB L2/L3 VSNs v3.3 • SPB Multicast v3.4 • SPB L2/L3 VSNs v7.1 • R/RS/8800 modules • SPB Multicast v7.2 Data Center 2 Core Server Access Virtual Services Fabric Server Server Server Server • Layer 2 Virtual Services Networks (VSNs) • Layer 3 Virtual Services Networks with VRF Extensions • Native IP Routing • Dual-homing into Fabric (SMLT – SPB) • Multicast over SPB • Virtual Services Fabric = Shortest Path Bridging (SPB) • IEEE 802.1aq SPBM • Uses IS-IS as the control plane • IS-IS adapted to support Layer 2, Layer 3, and Multicast Virtualization • Campus Core/Distribution using Switch Clustering (SMLT)
Ethertype B-VID Ethertype B-SA B-DA Ethernet Evolution – A Brief History Payload Payload Payload Ethertype Ethertype Payload C-VID C-VID Ethertype Ethertype Ethertype VID S-VID S-VID Ethertype Ethertype Ethertype Ethertype SA SA SA SA DA DA DA DA Service ID 802.1 802.1q 802.1ad Provider Bridges (Q-in-Q) I-SID SA = Source MAC address DA = Destination MAC address VID = VLAN ID C-VID = Customer VID S-VID = Service VID I-SID = Service ID B-VID = Backbone VID B-DA = Backbone DA B-SA = Backbone SA Regular Ethernet header 802.1ah Provider Backbone Bridges (PBB or MAC-in-MAC)
Virtualization in Action Backbone Edge Bridge Backbone Core Bridge C-VID | X | Y Payload C-VID | X | Y Payload A B C D E Host Y Host X Y X Payload C-VID | X | Y ISID | B-VID | A | E Payload PBB Header Edge VLAN ID (C-VID) mapped to ISID and PBB encapsulation added PBB encapsulation removed and original packet forwarded out Network Edge Network Core Network Edge Backbone Core Bridges have no awareness of Edge MAC Addresses
customer demarcs Service OAM (SID) UNI UNI Link Link Trunk OAM Link OAM Link OAM Link OAM Edge Transit Edge NNI NNI Switch Switch Switch Link Link Connectivity Fault Management (IEEE 802.1ag) • Functionality not previously available in Ethernet networks: • Fault Detection (failures) - Continuity Check Messages (CCM)* • Fault Verification (mis-configurations) - Loopback Messages (L2 Ping) • Fault Isolation (troubleshooting) - Link Trace Messages (L2 Traceroute) • Fault Notification (SNMP traps) • Hierarchy of Operational/Maintenance domains • Service layer : I-SID, Client (C-VLAN) • Trunk layer : B-VLAN, Port
Considerations when Migrating from SMLT to SPB • Switch Clustering (SMLT) • IP Routing – Global Routing Table • In-band Network Management using Global Routing Table • End-to-end Layer 2 VLANs • IPVPN-Lite • Minimising network disruption during migration
Migrating from SMLT to SPB • Typical Inception design use SMLT/ RSMLT with: • End-to-end L2 VLANs • IP Routing • IPVPN-Lite • Target design uses SPB and must migrate: • L2 VSNs • Native IP shortcuts • IPVPN-Lite over SPB or OptionalSPB L3 VSNs
8800G 8800A 8800C IST IST Tester Tester Tester Tester SMLT SMLT SMLT 8800D 8800B 8800G 8800A 8800C IST MLT SMLT 8800D 8800B Switch Clustering (SMLT) Inception: Network Edge Network Core Edge SMLT stays Target: Core SMLT Cluster (8800G-8800D) is decommissioned
8800G 8800A 3/11 3/2 3/5 3/1 MLT1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 10.0.2.0/24VLAN 1002 IST IST 3/29 4/20 3/30 4/1 4/30 Tester Tester Tester Tester 4/29 4/11 3/3 SMLT 3/1 4/29 4/12 3/2 8800D 8800B 8800G 8800A MLT1 8800C Two Core OSPF Vlans:Vlan 921 10.0.21.0/24Vlan 922 10.0.22.0/24 Two Core OSPF Vlans:Vlan 911 10.0.11.0/24Vlan 912 10.0.12.0/24 RSMLT IST RSMLT RSMLT SMLT 8800D 8800B IP Routing - Global Routing Table Inception: 10.0.2.0/24VLAN 1002 10.0.1.0/24 VLAN 1001 RSMLT + OSPF in Core Target: IS-IS (SPB) 10.0.1.0/24 VLAN 1001 Native IP Shortcuts over IS-IS
8800G 8800A 3/11 3/2 3/5 3/1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 IST IST 3/29 4/20 3/30 4/30 10.0.90.0/24VLAN 90 Tester Tester 4/29 4/11 3/3 3/1 4/29 4/12 3/2 8800D 8800B Two Core OSPF Vlans:Vlan 921 10.0.21.0/24Vlan 922 10.0.22.0/24 Two Core OSPF Vlans:Vlan 911 10.0.11.0/24Vlan 912 10.0.12.0/24 RSMLT RSMLT RSMLT In-band Network Management – Using IP GRT • ERS 8800 Routers managed via Circuitless IP (CLIP) in GRT • L2 Edge switches managed via edge Mgmt VLANs Inception: 10.0.0.10/32 CLIPs 10.0.0.11/32 CLIP 10.0.0.13/32 CLIPs 10.0.0.14/32 10.0.0.12/32
10.0.90.0/24VLAN 90 Tester Tester 8800G 8800A 3/1 8800C 3/30 IST MLT32 3/30 3/1 8800D 8800B In-band Network Management – Using IP GRT Target: • ERS 8800 Routers still managed via Circuitless IP (CLIP) in GRT • L2 Edge switches still managed via edge Mgmt VLANs • 8800G & 8800D only have single CLIP in GRT 10.0.0.10/32 CLIPs 10.0.0.11/32 CLIP 10.0.0.13/32 IS-IS (SPB) CLIPs 10.0.0.14/32 10.0.0.12/32
8800G 8800A SMLT 8800C SMLT IST IST Tester Tester Tester Tester SMLT 8800D 8800B VLAN 10 8800G 8800A SMLT 8800C IST MLT32 8800D 8800B Extending L2 VLANs Inception: I-SID 129010 Core has to do MAC learning and flooding No end-point provisioning Target: VLAN 10 SPB L2VSN VLAN 10
8800G 8800A 3/11 3/2 3/5 SMLT 8800C MLT1 3/12 3/3 4/30 3/22 3/29 3/21 IST IST 3/29 4/20 4/30 10.0.90.0/24VLAN 90 Tester Tester 4/29 4/11 3/3 4/29 4/12 3/2 SMLT 8800D 8800B Two Core OSPF Vlans:Vlan 921 10.0.21.0/24Vlan 922 10.0.22.0/24 Two Core OSPF Vlans:Vlan 911 10.0.11.0/24Vlan 912 10.0.12.0/24 RSMLT RSMLT RSMLT iBGP for IPVPN-Lite • iBGPpeerings run between CLIP interfaces over the GRT • CLIP reachability ensured by OSPF CLIP 10.0.0.11/32 Inception: IBGP Peering CLIP 10.0.0.13/32 IBGP Peering CLIP 10.0.0.12/32
10.0.90.0/24VLAN 90 Tester Tester 8800G 8800A 3/1 8800C 3/30 IST MLT 3/30 3/1 8800D 8800B iBGP for IPVPN-Lite CLIP 10.0.0.11/32 Target: • iBGP peering still run between CLIP interfaces over the GRT • There is no real change for BGP (and IPVPN-Lite) other than CLIP reachability which is now over IS-IS instead of OSPF IBGP Peering CLIP 10.0.0.13/32 IS-IS (SPB) IBGP Peering CLIP 10.0.0.12/32
8800G 8800A 3/11 3/2 3/5 MLT1 8800C MLT1 3/12 3/3 4/30 3/22 3/29 3/21 IST 3/29 4/20 4/1 4/30 Tester Tester Tester Tester 4/29 4/11 3/3 SMLT 4/29 3/2 8800D 8800B 8800G 8800A SMLT 8800C 3/12 Two Core OSPF Vlans:Vlan 921 10.0.21.0/24Vlan 922 10.0.22.0/24 Two Core OSPF Vlans:Vlan 911 10.0.11.0/24Vlan 912 10.0.12.0/24 3/29 RSMLT MLT32 RSMLT RSMLT 3/29 4/1 8800D 8800B Extending VRFs with IPVPN-Lite BGP IPVPN Inception: 10.1.102.0/24 10.1.101.0/24 VLAN 101 VLAN 102 BGP IPVPN IPVPN-Lite over SMLT Target 1: IS-IS (SPB) 10.1.101.0/24 10.1.102.0/24 VLAN 101 IPVPN-Lite over SPB VLAN 102
Tester Tester Tester Tester 8800G 8800G 8800A 8800A SMLT SMLT 8800C 8800C 3/12 3/3 3/22 3/29 3/21 MLT32 MLT32 3/29 4/20 4/1 4/1 4/30 4/11 3/3 8800D 8800D 8800B 8800B Extending VRFs with SPB L3VSNs BGP IPVPN Target 1: IS-IS (SPB) 10.1.101.0/24 10.1.102.0/24 VLAN 101 I-SID 10990001 VLAN 102 IPVPN-Lite over SPB Target 2: IS-IS (SPB) 10.1.101.0/24 10.1.102.0/24 VLAN 101 VLAN 102 SPB L3VSN; BGP is removed
Migration Without Disruption • Deploy SPB infrastructure in parallel • Verify SPB/IS-IS operation • Migrate Layer 2 VLANs to SPB Layer 2 VSNs • Migrate Global Routing Table IP Routing to SPB Native IP Shortcuts • Decommission OSPF, RSMLT, Core-SMLT
8800G 8800A 3/11 3/2 3/5 3/1 MLT1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 IST IST 3/29 4/20 3/30 4/1 4/30 Tester Tester 4/29 4/11 3/3 SMLT 3/1 4/29 4/12 3/2 MLT30 8800D 8800B IS-IS/SPB – Global Configuration Split-BEB Primary IS-IS Area 30.0000 spbm enable isisspbm 1 create isisspbm 1 nick-name f.00.13 isis spbm 1 add-b-vid 4001-4002 primary 4001 vlan 4001 create spbm-bvlan name "B-VLAN-1" vlan 4002 create spbm-bvlan name "B-VLAN-2" isis manual-area add 30.0000 isis sys-name 8800C isis enable Split-BEB Secondary spbm enable isisspbm 1 create isisspbm 1 nick-name f.00.12 isis spbm 1 add-b-vid 4001-4002 primary 4001 isis spbm 1 smlt peer-bmac 00:e0:7b:b4:b3:df isis spbm 1 smlt split-beb secondary vlan 4001 create spbm-bvlan name "B-VLAN-1" vlan 4002 create spbm-bvlan name "B-VLAN-2" isis manual-area add 30.0000 isis sys-name 8800B isis enable spbm enable isisspbm 1 create isisspbm 1 nick-name f.00.10 isis spbm 1 add-b-vid 4001-4002 primary 4001 isisspbm 1 smlt peer-bmac 00:14:0d:a0:13:df isisspbm 1 smlt split-beb primary vlan 4001 create spbm-bvlan name "B-VLAN-1" vlan 4002 create spbm-bvlan name "B-VLAN-2" isis manual-area add 30.0000 isis sys-name 8800G isis enable spbm enable isisspbm 1 create isisspbm 1 nick-name f.00.14 isisspbm 1 add-b-vid 4001-4002 primary 4001 isisspbm 1 smlt peer-bmac 00:0e:62:25:a3:df isis spbm 1 smlt split-beb secondary vlan 4001 create spbm-bvlan name "B-VLAN-1" vlan 4002 create spbm-bvlan name "B-VLAN-2" isis manual-area add 30.0000 isis sys-name 8800D isis enable spbm enable isisspbm 1 create isisspbm 1 nick-name f.00.11 isis spbm 1 add-b-vid 4001-4002 primary 4001 isis spbm 1 smlt peer-bmac 00:0f:cd:57:e3:df isisspbm 1 smlt split-beb primary vlan 4001 create spbm-bvlan name "B-VLAN-1" vlan 4002 create spbm-bvlan name "B-VLAN-2" isis manual-area add 30.0000 isis sys-name 8800A isis enable SPBM SMLT is not needed on this cluster but has to be configured to enable ISIS because an IST exists; this will be redundant later on once the IST is deleted
8800G 8800A 3/11 3/2 3/5 3/1 MLT1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 IST IST 3/29 4/20 3/30 4/1 4/30 Tester Tester 4/29 4/11 3/3 SMLT 3/1 4/29 4/12 3/2 MLT30 8800D 8800B IS-IS/SPB – Interface Configuration ethernet 4/29,4/30 isis create ethernet 4/29,4/30 isisspbm 1 state enable ethernet 4/29,4/30 isis enable ethernet 3/2,3/3 isis create ethernet 3/2,3/3 isisspbm 1 state enable ethernet 3/2,3/3 isis enable mlt 32 isis create mlt 32 isisspbm 1 state enable mlt 32 isis enable ethernet 3/5,3/11,3/12 isis create ethernet 3/5,3/11,3/12 isis spbm 1 state enable ethernet 3/5,3/11,3/12 isis enable mlt 32 isis create mlt 32 isis spbm 1 state enable mlt 32 isis enable ethernet 3/2,3/3 isis create ethernet 3/2,3/3 isis spbm 1 state enable ethernet 3/2,3/3 isis enable mlt 32 isis create mlt 32 isis spbm 1 state enable mlt 32 isis enable ethernet 4/11,4/12,4/29 isis create ethernet 4/11,4/12,4/29 isisspbm 1 state enable ethernet 4/11,4/12,4/29 isis enable mlt 32 isis create mlt 32 isisspbm 1 state enable mlt 32 isis enable
8800G 8800A 3/11 3/2 3/5 3/1 MLT1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 IST IST 3/29 4/20 3/30 4/1 4/30 Tester Tester 4/29 4/11 3/3 SMLT 3/1 4/29 4/12 3/2 MLT30 8800D 8800B Connectivity Fault Management Configuration cfmmd "spbm" create index 1 cfmmd "spbm" ma "bvid" create index 1 cfmmd "spbm" ma "bvid" mep 1 create state enable cfmmd "spbm" ma "bvid" mep 2 create state enable vlan 4001 add-nodal-mep spbm.bvid.1 vlan 4001 add-nodal-mip-level 4 vlan 4002 add-nodal-mep spbm.bvid.2 vlan 4002 add-nodal-mip-level 4 cfm md "spbm" create index 1 cfm md "spbm" ma "bvid" create index 1 cfm md "spbm" ma "bvid" mep 1 create state enable cfm md "spbm" ma "bvid" mep 2 create state enable vlan 4001 add-nodal-mep spbm.bvid.1 vlan 4001 add-nodal-mip-level 4 vlan 4002 add-nodal-mep spbm.bvid.2 vlan 4002 add-nodal-mip-level 4 cfm md "spbm" create index 1 cfm md "spbm" ma "bvid" create index 1 cfm md "spbm" ma "bvid" mep 1 create state enable cfm md "spbm" ma "bvid" mep 2 create state enable vlan 4001 add-nodal-mep spbm.bvid.1 vlan 4001 add-nodal-mip-level 4 vlan 4002 add-nodal-mep spbm.bvid.2 vlan 4002 add-nodal-mip-level 4 cfm md "spbm" create index 1 cfm md "spbm" ma "bvid" create index 1 cfm md "spbm" ma "bvid" mep 1 create state enable cfm md "spbm" ma "bvid" mep 2 create state enable vlan 4001 add-nodal-mep spbm.bvid.1 vlan 4001 add-nodal-mip-level 4 vlan 4002 add-nodal-mep spbm.bvid.2 vlan 4002 add-nodal-mip-level 4 cfm md "spbm" create index 1 cfm md "spbm" ma "bvid" create index 1 cfm md "spbm" ma "bvid" mep 1 create state enable cfm md "spbm" ma "bvid" mep 2 create state enable vlan 4001 add-nodal-mep spbm.bvid.1 vlan 4001 add-nodal-mip-level 4 vlan 4002 add-nodal-mep spbm.bvid.2 vlan 4002 add-nodal-mip-level 4
Migration Without Disruption • Deploy SPB infrastructure in parallel • Verify SPB/IS-IS Operation • Migrate Layer 2 VLANs to SPB Layer 2 VSNs • Migrate Global Routing Table IP Routing to SPB Native IP Shortcuts • Decommission OSPF, RSMLT, Core-SMLT
8800G 8800A 3/11 3/2 3/5 3/1 MLT1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 IST IST 3/29 4/20 3/30 4/1 4/30 Tester Tester 4/29 4/11 3/3 SMLT 3/1 4/29 4/12 3/2 MLT30 8800D 8800B CFM – Checking Operation 8800C:5# l2ping 4001.8800A Please wait for l2ping to complete or press any key to abort ----00:e0:7b:b4:b3:df L2 PING Statistics---- 0(68) bytes of data 1 packets transmitted, 1 packets received, 0.00% packet loss round-trip (us) min/max/ave/stdv = 545/545/545.00/ 0.00 8800C:5# l2ping 4002.8800A Please wait for l2ping to complete or press any key to abort ----00:e0:7b:b4:b3:df L2 PING Statistics---- 0(68) bytes of data 1 packets transmitted, 1 packets received, 0.00% packet loss round-trip (us) min/max/ave/stdv = 547/547/547.00/ 0.00 8800C:5# l2ping 4001.8800B Please wait for l2ping to complete or press any key to abort ----00:0f:cd:57:e3:df L2 PING Statistics---- 0(68) bytes of data 1 packets transmitted, 1 packets received, 0.00% packet loss round-trip (us) min/max/ave/stdv = 558/558/558.00/ 0.00 8800C:5# l2ping 4002.8800B Please wait for l2ping to complete or press any key to abort ----00:0f:cd:57:e3:df L2 PING Statistics---- 0(68) bytes of data 1 packets transmitted, 1 packets received, 0.00% packet loss round-trip (us) min/max/ave/stdv = 517/517/517.00/ 0.00
Migration Without Disruption • Deploy SPB infrastructure in parallel • Verify SPB/IS-IS operation • Migrate Layer 2 VLANs to SPB Layer 2 VSNs • Migrate Global Routing Table IP Routing to SPB Native IP Shortcuts • Decommission OSPF, RSMLT, Core-SMLT
Avoiding Migration Loops !!! • There is no Spanning Tree over SMLT • There is no Spanning Tree over SPB L2VSNs • It would be quite easy to create a loop between old and new transport • To prevent this from happening ERS 8800 enforces a consistency check: • An ISID cannot be assigned to a VLAN if that VLAN exists on ISIS interfaces SMLT Core VLAN 10 I-SID 12990010 Broadcast Edge VLAN 10 Edge VLAN 10 SPB Core
Migrating Extended VLAN 10Step #1 • First delete VLAN 10 on uplinks to Core • Traffic on VLAN 10 will be interrupted SMLT Core VLAN 10 I-SID 12990010 Edge VLAN 10 Edge VLAN 10 SPB Core
Migrating Extended VLAN 10Step #2 • Re-configure VLAN 10 at the Edge to use SPB L2 VSN • vlan 10 i-sid 12990010 • Traffic on VLAN 10 will now resume almost immediately SMLT Core VLAN 10 I-SID 12990010 Edge VLAN 10 Edge VLAN 10 SPB Core
Migrating Extended VLAN 10Step #3 • VLAN 10 in the SMLT Core is no longer needed • Verify that no more traffic is using it • Then delete it SMLT Core VLAN 10 I-SID 12990010 Edge VLAN 10 Edge VLAN 10 SPB Core
Migration Without Disruption • Deploy SPB infrastructure in parallel • Verify SPB/IS-IS operation • Migrate Layer 2 VLANs to SPB Layer 2 VSNs • Migrate Global Routing Table IP Routing to SPB Native IP Shortcuts • Decommission OSPF, RSMLT, Core-SMLT
Migrating GRT IP Routing from OSPF to IS-IS • There is no risk of loops here • There is no traffic interruption during migration • Simply enable SPB Native IP shortcuts • IS-IS routes, by default, have a higher pref (7) than OSPF routes (20) • The default IS-IS pref must be temporarily increased beyond OSPF to ensure that IP routes are correctly learned by IS-IS before actually switching over to IS-IS • Once satisfied that IS-IS routes are good, then restore IS-IS default pref to switch over to IS-IS OSPF + RSMLT 10.0.1.0/24 10.0.2.0/24 IS-IS
Migration Without Disruption • Deploy SPB infrastructure in parallel • Verify SPB/IS-IS operation • Migrate Layer 2 VLANs to SPB Layer 2 VSNs • Migrate Global Routing Table IP Routing to SPB Native IP Shortcuts • Decommission OSPF, RSMLT, Core-SMLT
Decommissioning OSPF & RSMLT • OSPF (and RSMLT) no longer needed • Disable OSPF • Delete Core OSPF VLANs OSPF + RSMLT 10.0.1.0/24 10.0.2.0/24 IS-IS
8800G 8800A 3/11 3/2 3/5 3/1 MLT1 8800C MLT1 3/12 3/3 4/30 3/30 3/22 3/29 3/21 IST IST 3/29 4/20 3/30 4/1 4/30 Tester Tester 4/29 4/11 3/3 SMLT 3/1 4/29 4/12 3/2 MLT30 8800D 8800B Decommissioning OSPF & RSMLT • There are no more OSPF routes in the IP routing table • Hence deleting OSPF has no impact on traffic IS-IS (SPBM) 10.0.2.0/24VLAN 1002 10.0.1.0/24 VLAN 1001 ip ospf disable vlan 921 delete vlan 922 delete ip ospf disable vlan 911 delete vlan 912 delete vlan 921 delete vlan 922 delete ipospf disable vlan 911 delete vlan 912 delete vlan 921 delete vlan 922 delete ip ospf disable vlan 911 delete vlan 912 delete ip ospf disable vlan 911 delete vlan 912 delete
Decommissioning Core-SMLT • At this point there should be no more VLANs running on the Core SMLT Cluster • After verifying this • Decommission SMLT and the IST on the Core nodes • The Core MLT instances can also be deleted SMLT 10.0.1.0/24 10.0.2.0/24 IS-IS
Avaya VENA Campus Network Architecture with SPB Avaya ERS 8800 Avaya VSP 9000 Data Center 1 Core Server Access Avaya VSP 7000 Campus Core Distribution (optional) • SPB10.2 (Q3 12) • Layer 2 VSNs • Lossless - 3.2 • SPB L2/L3 VSNs - 3.3 • SPB Multicast - 3.4 • SPB L2/L3 VSNs - 7.1 • R/RS/8800 modules • SPB Multicast - 7.2 Data Center 2 Core Server Access Virtual Services Fabric Server Server Server Server • Layer 2 Virtual Services Networks (VSNs) • Layer 3 Virtual Services Networks with VRF Extensions • Native IP Routing • Dual-homing into Fabric (SMLT – SPB) • Multicast over SPB