380 likes | 605 Views
Ethernet connectivity and options. Unit objectives. After completing this unit, you should be able to: List the BladeCenter components required for Ethernet connectivity Name the management components supporting Ethernet switch modules
E N D
Unit objectives After completing this unit, you should be able to: • List the BladeCenter components required for Ethernet connectivity • Name the management components supporting Ethernet switch modules • Select the Ethernet I/O switch module to meet a specific requirement • List the components required to provide additional Ethernet connectivity to a blade server • Identify the method used to provide blade server Ethernet connection redundancy
BladeCenter chassis I/O overview • BladeCenter chassis I/O overview • Internal chassis traffic flow • Switch I/O module management connectivity • BNT (Nortel) Ethernet switch I/O modules • Server connectivity module • Pass-thru module • High speed I/O modules overview • Ethernet expansion cards
BladeCenter chassis overview: Switching path BladeCenter Chassis Physical Components
Chassis overview: Gigabit Ethernet routing SW Module 1 1………..14 SW Module 3 1………..14 SERDES Ethernet Line 1 Midplane (Upper Section) Line 3 Network Interface Expansion Card Line 4 Midplane (Lower Section) Line 2 SERDES Ethernet Processor blade #1 1………..14 SW Module 4 1………..14 SW Module 2
Chassis overview: Ethernet port assignments (1 of 2) Top ESM ( Bay 1 ) ESM External uplinks 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Mgt 1 Mgt 2 Eth 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 MM 1 Eth 0 Uplink MM 1 B B B B B B B B B B B B B B S S S S S S S S S S S S S S 1 1 1 1 1 0 1 2 3 4 1 2 3 4 5 6 7 8 9 MM 2 Eth 0 MM 2 Uplink 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Eth 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Mgt 1 Mgt 2 ESM External uplinks Bottom ESM ( Bay 2 ) BladeCenter Chassis
Chassis overview: Ethernet port assignments (2 of 2) Top ESM ( Bay 1 ) ESM External uplinks 1 2 3 4 5 6 8 9 10 11 12 13 Mgt 1 Eth 1 1 1 1 1 1 1 MM 1 Eth 0 Uplink MM 1 B B B B B B S S S S S S 1 2 3 4 5 6 2 2 2 2 2 2 BladeCenter S Chassis
Internal chassis traffic flow • BladeCenter chassis Ethernet overview • Internal chassis traffic flow • Switch I/O module management connectivity • BNT (Nortel) Ethernet switch I/O modules • Server connectivity module • Pass-Thru module • High speed I/O modules overview • Ethernet expansion cards
Ethernet connectivity: Layer 2 traffic flow Hidden internal filter that blocks traffic flow between the External ports and Mgmt Mod ports. Prevents STP loops between ESM uplinks and MM uplinks (as well as between ESMs) Upstream connections Upstream connections Red = Ethernet External ports MM1 Management Module Ports CPU Ethernet Switch Module (ESM) MM2 14 Internal Blade ports BladeCenter Chassis
Traffic flow: NIC teaming and trunk failover If failure anywhere on the link toward the upstream switch, the NIC on blade server does not know about the failure and may continue to send traffic toward the top switch, which will discard the traffic. The Trunk Failover feature addresses this issue. 1 2 3 4 5 6 1 2 3 4 5 6 x ESM ESM X X VLAN X Teamed Active/Standby If the switch fails in such a way that the link toward the Blade server goes down, or NIC fails, Blade server can sense this and redirect traffic out the other NIC toward the bottom switch. NIC Teaming can take care of this without the need for trunk failover. NIC 2 NIC 1 Logical NIC Interface Blade Server
Traffic flow: Trunk failover example 1 • If Po1 goes down, trunk failover takes over and shuts down internal downstream defined port(s) • This alerts NIC Teaming to an upstream failure at which point Teaming switches to the other path out of the server • This example shows a single VLAN to Teamed NICs, It is possible to also carry multiple VLANs to the Teamed NICs 1 2 3 4 5 6 1 2 3 4 5 6 • Po1 VLAN X Must be carried on all uplinks from ESM • Po2 • Po2 VLAN X Must be carried between ESMs via an external path • Link state group1 Upstream • Link state group1 Upstream • Po1 • Po1 ESM ESM VLAN X Teamed Active/Standby • Link state group Downstream • Link state group Downstream • NIC Teaming NIC 2 NIC 1 Logical NIC Interface Blade Server
Traffic flow: Trunk failover example 2 • If only one port channel goes down, spanning tree allows backup path to other port channel • If both port channels go down, trunk failover takes over and shuts down internal downstream defined port(s) • This alerts NIC Teaming to an upstream failure at which point it switches to the other CIGESM • This example shows a single VLAN to Teamed NICs, It is possible to also carry multiple VLANs to the Teamed NICs 1 2 3 4 5 6 1 2 3 4 5 6 • Po1 VLAN X Must be carried on all uplinks from ESM • Po2 • Po3 • Po2 • Po3 • Link state group1 Upstream • Link state group1 Upstream • Po1 • Po2 • Po1 • Po2 ESM ESM VLAN X Teamed Active/Standby • Link state group1 Downstream • Link state group1 Downstream • NIC Teaming NIC 2 NIC 1 Logical NIC Interface Blade Server
Switch I/O module management connectivity • BladeCenter chassis Ethernet overview • Internal chassis traffic flow • Switch I/O module management connectivity • BNT (Nortel) Ethernet switch I/O modules • Server connectivity module • Pass-Thru module • High speed I/O modules overview • Ethernet expansion cards
Management connectivity for the ESM Management Workstation Legend Ethernet Ethernet path 2 I2C interface I2C path 1 3 Serial Serial path Routed Production Network Management Network 3 4 5 2 Web Interface Ethernet Switch Module 1A Management Module Blade Server 1B 4
BNT (Nortel) Ethernet switch I/O modules • BladeCenter chassis Ethernet overview • Internal chassis traffic flow • Switch I/O module management connectivity • BNT (Nortel) Ethernet switch I/O modules • Server connectivity module • Pass-Thru module • High speed I/O modules overview • Ethernet expansion cards
BNT GbESM switch I/O modules: Layer 2-3 • Layer 2/3 copper GbESM • 14 internal 1 Gb FDX connections • 6 external 1 Gb FDX connections • 2 10/100 Mb FDX management ports • Layer 2/3 Fibre GbESM • 14 internal 1 Gb FDX connections • 6 external 1 Gb FDX fibre (SX) connections • 2 10/100 FDX management ports BNT Copper and Fibre Layer 2-3 Switch I/O Modules
BNT L2/3 GbESM: Simplified block diagram Management Processor Control Application Switch connector Gigabit Ethernet Accelerator Data Memory 6 external 10/100/1000 Mbps Copper Ports (Auto-negotiation enabled) Or 6 external 10/100/1000 Mbps Copper Ports (Auto-negotiation enabled) And 14 internal 1000 Mbps server links 2 internal 100 Mbps management links
BNT L2/3 GbESM: Network interfaces Blade 12 Blade 13 Blade 14 Blade 3 Blade 1 Blade 2 Default VLAN VLAN 1 Management VLAN VLAN 4095 IP Interface 128 Int3 (3) Int12 (12) Int13 (13) Int14 (14) Int1 (1) Int2 (2) Ext1 (17) Ext2 (18) Mgt1 (15) Management Module 1 Customer Network Management Module 2 Mgt2 (16) Ext5 (21) Ext6 (22)
BNT Nortel layer 2-7 GbESM Layer 5-7 Layer 4 Layer 2/3 • The Layer 2-7 GbESM features: • Support for up to 300,000 simultaneous Layer 2-7 sessions with full wire-speed packet forwarding for all connections • Link aggregation on the four external ports • Layer 4 load balancing for computing optimization • Virtual router redundancy to facilitate high application availability in the event of application, operating system or hardware failures
Combined 1 Gb and 10 Gb external ports Three slots for 10 Gb Ethernet SFP+ modules Six 1 Gb RJ45 ports Fourteen internal 1 Gb ports BNT 1/10 Gb uplink Ethernet switch module
BladeCenter chassis Ethernet overview Internal chassis traffic flow Switch I/O module management connectivity BNT (Nortel) Ethernet switch I/O modules Server connectivity module Pass-Thru module High speed I/O modules overview Expansion cards Server connectivity module
Server connectivity module: Product highlights • Low-cost Layer 2 device • Simplified user interface to enable server administrator control of device • Ideal for SMB customers and applications • Function summary • Basic Layer 2 connectivity within chassis • “Non-networking”, server admin setup and control • Simple, GUI/CLI interface • Create blade groups (simple VLANs) • Create external port groups (auto Etherchannel) • Enable trunk failover (redundancy option) • Set external port speed • Assign tagged VLANs to blade groups
Pass-Thru module • BladeCenter chassis Ethernet overview • Internal chassis traffic flow • Switch I/O module management connectivity • BNT (Nortel) Ethernet switch I/O modules • Server connectivity module • Pass-Thru module • High speed I/O modules overview • Expansion cards
Intelligent copper pass-thru module: Product highlights • 14 external RJ-45 copper ports for making connections to the external network. • The external ports can operate at 100 or 1000 Mbps. • 14 internal bi-directional ports, connected to each of the blade servers • One-to-one mapping of internal Ethernet ports on the blade to external copper RJ-45 • GUI based operator interface
High speed I/O modules overview • BladeCenter chassis Ethernet overview • Internal chassis traffic flow • Switch I/O module management connectivity • BNT (Nortel) Ethernet switch I/O modules • Server connectivity module • Pass-Thru module • High speed I/O modules overview • Expansion cards
High speed I/O modules: Overview BNT 10Gb Ethernet Switch I/O Module BladeCenter H and HT High Speed Module 10 External Ports 14 Internal Ports BNT 10Gb Ethernet Pass-Thru I/O Module BladeCenter H and HT High Speed Module 14 External Ports 14 Internal Ports BNT 10Gb Ethernet Switch I/O Module BNT 10Gb Ethernet Pass-Thru I/O Module
BladeCenter chassis Ethernet overview Internal chassis traffic flow Switch I/O module management connectivity BNT (Nortel) Ethernet switch I/O modules Server connectivity module Pass-Thru module High speed I/O modules overview Expansion cards Expansion cards
PCI Express host interface Broadcom BCM5709S communication module BladeCenter Open Fabric Manager (BOFM) support Connection to 1000BASE-X environments using BladeCenter Ethernet switches Ethernet expansion cards: Combination I/O vertical (CIOv) CIOv Ethernet Expansion Card IBM HS22 Blade Server with Ethernet CIOv card installed
Ethernet expansion cards: Combination form factor vertical (CFFv) • Based on Broadcom 5704 chipset • Two 1 Gb ports per blade server IBM HS22 Blade Server with Ethernet CFFv card installed Ethernet CFFv Expansion Card
Ethernet expansion cards: QLogic iSCSI • Dual ports • TCP/IP Offload Engine • TC/IP stack implemented in firmware • Provides iSCSI diskless boot capability QLogic iSCSI StFF Expansion Card QLogic iSCSI Management Utility
Ethernet expansion cards: High speed overview • High Speed Form horizontal (CFFh) expansion cards • Interface to Horizontally mounted I/O modules • BladeCenter H and HT chassis • PCI-E bus connections QLogic 2-Port 10Gb Converged Network Adapter CFFh IBM BladeCenter 2/4-Port CFFh card Broadcom 10Gb 4-Port CFFh card NetXen 10Gb 2-Port CFFh card
Key words • Device Manager • Failover • Fibre Ethernet • Graphical user interface (GUI) • Local Area Network (LAN) • Layer 2 – 7 • Media Access Control (MAC) Address • Power On Self Test (POST) • Spanning Tree • Serializer/deserializer (SERDES) • Server Connectivity Module (SCM) • Open Shortest Path First (OSPF) • Uplink • Virtual Local Area Network (VLAN) tagging • Virtual Router Redundancy Protocol (VRRP) • Trunking
True/False: In order to provide Ethernet connectivity, each blade server requires an Ethernet expansion card. A customer requires policy management capability in the application through Transport layers of the OSI model. Select the switch module appropriate for these requirements. BNT 10Gb Pass-thru Module BNT L2/3 Copper GbESM IBM Server Connectivity Module BNT L2-7 GbESM True/False:The Ethernet switch I/O modules can be managed through the AMM management network, or through the switch I/O module external ports Checkpoint (1 of 2)
True/False: In order to provide Ethernet connectivity, each blade server requires an Ethernet expansion card. Answer: False A customer requires policy management capability in the application through Transport layers of the OSI model. Select the switch module appropriate for these requirements. BNT 10Gb Pass-thru Module BNT L2/3 Copper GbESM IBM Server Connectivity Module BNT L2-7 GbESM Answer: d True/False:The Ethernet switch I/O modules can be managed through the AMM management network, or through the switch I/O module external ports Answer: True Checkpoint solutions (1 of 2)
In order to support 4 Ethernet connections from a Blade server, which of the following is correct? An expansion blade is required Add an Ethernet expansion card and switch module in Bay 2 Add an Ethernet expansion card and switch modules in Bays 3 and 4 Add an IBM Server connectivity module True/False:NIC Teaming can be used to provide redundancy, utilizing the paths from the blade server onboard NICs through the midplane to Ethernet switch I/O modules in Bays 1 and 2 of the chassis. Checkpoint (2 of 2)
In order to support 4 Ethernet connections from a blade server, which of the following is correct? An expansion blade is required Add an Ethernet expansion card and switch module in Bay 2 Add an Ethernet expansion card and switch module in Bays 3 and 4 Add an IBM Server connectivity module Answer: c True/False:NIC Teaming can be used to provide redundancy, utilizing the paths from the blade server onboard NICs through the midplane to Ethernet switch I/O modules in Bays 1 and 2 of the chassis. Answer: True Checkpoint solutions (2 of 2)
Unit summary Having completing this unit, you should be able to: • List the BladeCenter components required for Ethernet connectivity • Name the management components supporting Ethernet switch modules • Select the Ethernet I/O switch module to meet a specific requirement • List the components required to provide additional Ethernet connectivity to a blade server • Identify the method used to provide blade server Ethernet connection redundancy