Cisco Catalyst 6500 IOS Update

1. Cisco Catalyst 6500 IOS Update Chew Kin Pheng, Systems Engineer (kchew@cisco.com)

2. Introduction Embedded Event Monitoring (EEM) Generic Online Diagnostics (GOLD) Smart Call Home (SCH) Global Balancing Protocol (GLBP) Agenda

3. 12.2(33)SXH Software SHIPPING! 200+ Features with Full IOS Software Modularity NEW CatOS to IOS Transition Release Major Security Enhancements (IBNS, 802.1x etc) Virtual Switching & L2 Scalability Innovations Continued End-To-End Leadership Wiring Closet Backbone Data Center EWAN Metro SPNetwork • LLDP-MED • NAC Integration • IPv6 Innovations • 16 port 10G linecard • VS-S720-10G • IPsec Leadership • Multicast VPN Inter-AS and Extranet • LLDP-MED • NAC Integration • IOS Modularity • GOLD • CPP • Enhanced Object Tracking • HSRP and GLBP SSO • 16-way Loadbalancing • Fast Fabric Switchover • IOS Software Modularity • BFD with BGP • MPLS HA • MPLS FRR link and Node protection • Multiplexed UNI • Smart Call Home • Smart-Ports • AutoSecure • Multiple SPAN Enhancements • EEM • Smart Call-Home • EEM • IP SLA • Smart Call-Home • E-OAM (802.1ag and 802.3ah) • MPLS MIBs • Multi-VRF with Multicast • 802.1x, MAC Auth, Web Auth for Access Control • Smart Call Home • Smart-Ports, AutoQoS, AutoSecure • VRF Aware Services • L2, L3 VPN Innovations • MPLS (L2, L3VPN, TE) Innovations • VRF Aware Services • Private Hosts • NBAR on PISA • AutoQoS • Per interface NDE • NetFlow Top Talkers • Multcast NDE • NetFlow Top Talkers • Per interface NDE • Sophisticated QOS support with LLQ, cRTP, LFI, MLPPP • Sophisticated QOS support for optimized Triple Play services • FPM on PISA • CIST, NAC, IBNS Solution Integration • Policy-Based ACLs • IGMP Filtering • Policy-Based ACLs • Multicast Router Guard • 16K IPSec tunnels • DMVPN support in HW • Layer 3 NAC • Address Spoofing Prevention • CoPP

4. Embedded Event Management(EEM) Overview

5. EEM – What is it? • Embedded Event Manager (EEM) is a programmable subsystem that is present in the IOS that runs on the Catalyst 6500 • It allows Network Administrators to automate responses to specific events that occur on the switch Simplified Operation - Embedded Event Manager provides a means to automate the operational management in real time - EEM monitors for specific events on the switch and can invoke pre defined actions to correct, take remedial action and report the event to network operations…

6. EEM - How does it work?

7. EEM Basic Architecture

8. EEM - Examples of its Use?

9. EEM - Examples of its Use?

10. Catalyst 6500 ManagementSimplified Operation - EEM Example Automate switch configuration for connected IP phones

11. EEM - The Hardware and Software it works with?

12. Generic Online Diagnostics For The Catalyst 6500

13. Generic Online Diagnostics What is GOLD? • GOLD defines a common framework for diagnostics operations across Cisco platforms running Cisco IOS Software. • Goal: check the health of hardware components and verify proper operation of the system data plane and control plane at run-time and boot-time. • Provides a common CLI and scheduling for field diagnostics including : • Bootup tests (includes online insertion) • Health monitoring tests (background non-disruptive) • On-Demand tests (disruptive and non-disruptive) • User scheduled tests (disruptive and non-disruptive) • CLI access to data via management interface

14. Generic Online DiagnosticsHow does GOLD work? • Diagnostic packet switching tests verify that the system is operating correctly: • Is the supervisor control plane and forwarding plane functioning properly? • Is the standby supervisor ready to take over? • Are linecards forwarding packets properly? • Are all ports working? • Is the backplane connection working? • Other types of diagnostics tests including memory and error correlation tests are also available Forwarding Engine Linecard Fabric Forwarding Engine CPU Active Supervisor Standby Supervisor Linecard

15. Generic Online DiagnosticsWhat type of failure does GOLD detect? • Diagnostics capabilities built in hardware • Depending on hardware, GOLD can catch: • Port Failure • Bent backplane connector • Bad fabric connection • Malfunctioning Forwarding engines • Stuck Control Plane • Bad memory • …

16. Generic Online DiagnosticsDiagnostic Integration Configuration/reporting Action Boot-up diagnostics Runtime diagnostics • Default corrective action • Supervisor reset Supervisor switch-over • Fabric switch-over • Port shut down • Line card reset • Line card power down • Generate a call-home message • Trigger Syslog • Trigger EEM policies • Generate SNMP Trap On-demand Configure online diagnostics and check diagnostics results Scheduled Health-monitoring Provides generic diagnostics framework Automated action based on diagnostics results Verify hardware functionalities Detect and identify problems before they result in network downtime!

17. Generic Online DiagnosticsDiagnostic Operation Run During System Bootup, Line Card OIR or Supervisor Switchover Makes Sure Faulty Hardware Is Taken out of Service Boot-Up Diagnostics Switch(config)#diagnostic bootup level complete Runtime Diagnostics Health-Monitoring Non-Disruptive Tests Run in the Background Serves as HA Trigger Switch(config)#diagnostic monitor module 5 test 2 Switch(config)#diagnostic monitor interval module 5 test 2 00:00:15 On-Demand Switch#diagnostic start module 4 test 8 Module 4: Running test(s) 8 may disrupt normal system operation Do you want to continue? [no]: y Switch#diagnostic stop module 4 All Diagnostics Tests Can Be Run on Demand, for Troubleshooting Purposes. It Can Also Be Used As A Pre-deployment Tool Scheduled Switch(config)#diagnostic schedule module 4 test 1 port 3 on Jan 3 2005 23:32 Switch(config)#diagnostic schedule module 4 test 2 daily 14:45 Schedule Diagnostics Tests, for Verification and Troubleshooting Purposes

18. Switch#show diagnostic content mod 5 Module 5: Supervisor Engine 720 (Active) <snip> Testing Interval ID Test Name Attributes (day hh:mm:ss.ms) ==== ================================== ============ ================= 1) TestScratchRegister -------------> ***N****A*** 000 00:00:30.00 2) TestSPRPInbandPing --------------> ***N****A*** 000 00:00:15.00 3) TestTransceiverIntegrity --------> **PD****I*** not configured 4) TestActiveToStandbyLoopback -----> M*PDS***I*** not configured 5) TestLoopback --------------------> M*PD****I*** not configured 6) TestNewIndexLearn ---------------> M**N****I*** not configured 7) TestDontConditionalLearn --------> M**N****I*** not configured 8) TestBadBpduTrap -----------------> M**D****I*** not configured 9) TestMatchCapture ----------------> M**D****I*** not configured 10) TestProtocolMatchChannel --------> M**D****I*** not configured 11) TestFibDevices ------------------> M**N****I*** not configured 12) TestIPv4FibShortcut -------------> M**N****I*** not configured 13) TestL3Capture2 ------------------> M**N****I*** not configured 14) TestIPv6FibShortcut -------------> M**N****I*** not configured 15) TestMPLSFibShortcut -------------> M**N****I*** not configured 16) TestNATFibShortcut --------------> M**N****I*** not configured 17) TestAclPermit -------------------> M**N****I*** not configured 18) TestAclDeny ---------------------> M**N****A*** 000 00:00:05.00 19) TestQoSTcam ---------------------> M**D****I*** not configured <snip> Generic Online DiagnosticsView the GOLD Tests and Attributes Diagnostics test suite attributes: M/C/* - Minimal bootup level test / Complete bootup level test / NA B/* - Basic ondemand test / NA P/V/* - Per port test / Per device test / NA D/N/* - Disruptive test / Non-disruptive test / NA S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA E/* - Always enabled monitoring test / NA A/I - Monitoring is active / Monitoring is inactive R/* - Power-down line cards and need reset supervisor / NA K/* - Require resetting the line card after the test has completed / NA T/* - Shut down all ports and need reset supervisor / NA

19. 20) TestL3VlanMet -------------------> M**N****I*** not configured n/a 21) TestIngressSpan -----------------> M**N****I*** not configured n/a 22) TestEgressSpan ------------------> M**D****I*** not configured n/a 23) TestNetflowInlineRewrite --------> C*PD****I*** not configured n/a 24) TestFabricSnakeForward ----------> M**N****I*** not configured n/a 25) TestFabricSnakeBackward ---------> M**N****I*** not configured n/a 26) TestTrafficStress ---------------> ***D****I**T not configured n/a 27) TestFibTcamSSRAM ----------------> ***D*X**IR** not configured n/a 28) TestAsicMemory ------------------> ***D*X**IR** not configured n/a 29) TestNetflowTcam -----------------> ***D*X**IR** not configured n/a 30) ScheduleSwitchover --------------> ***D****I*** not configured n/a 31) TestFirmwareDiagStatus ----------> M**N****I*** not configured n/a 32) TestAsicSync --------------------> ***N****A*** 000 00:00:15.00 10 Generic Online DiagnosticsGOLD Test Attributes (Con’t) Diagnostics test suite attributes: M/C/* - Minimal bootup level test / Complete bootup level test / NA B/* - Basic ondemand test / NA P/V/* - Per port test / Per device test / NA D/N/* - Disruptive test / Non-disruptive test / NA S/* - Only applicable to standby unit / NA X/* - Not a health monitoring test / NA F/* - Fixed monitoring interval test / NA E/* - Always enabled monitoring test / NA A/I - Monitoring is active / Monitoring is inactive R/* - Power-down line cards and need reset supervisor / NA K/* - Require resetting the line card after the test has completed / NA T/* - Shut down all ports and need reset supervisor / NA Pay Extra Attention to Memory Tests: Memory Tests Can Take Hours to Complete and a Reset Is Required After Running These Tests

20. MSFC Port ASIC PFC3 RP CPU L3/4Engine SP CPU FabricInterface/ ReplicationEngine L2 Engine Switch Fabric DBUS RBUS 16 GbpsBus EOBC Generic Online Diagnostics An example: Supervisor datapath coverage Monitors forwarding path between the Switch Processor, Route Processor and Forwarding Engine Runs Periodically every 15 Seconds after System is Online (Configurable) 10 Consecutive Failures is treated as FATAL and will result in supervisor switchover or supervisor reset Switch(config)#diagnostic monitor module 5 test 2 Switch(config)#diagnostic monitor interval module 5 test 2 00:00:15

21. Switch#show diagnostic result mod 7 Current bootup diagnostic level: complete Module 7: CEF720 24 port 1000mb SFP Overall Diagnostic Result for Module 7 : MINOR ERROR Diagnostic level at card bootup: complete Test results: (. = Pass, F = Fail, U = Untested) 1) TestTransceiverIntegrity: Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ---------------------------------------------------------------------------- U U . U . . U U . . U U . . U U U U U U U U U U 2) TestLoopback: Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ---------------------------------------------------------------------------- . . . . . . . . . . . . F . . . . . . . . . . . 3) TestScratchRegister -------------> . 4) TestSynchedFabChannel -----------> . <snip> Generic Online DiagnosticsView GOLD Results

22. GOLD Operation Example GOLD generic Syslog messages start with the string “DIAG”; CONST_DIAG” messages platform specific… Bootup Test Failure: • %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 2: TestL3VlanMet failed Health Monitoring Test Failure: • %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 5 TestSPRPInbandPing consecutive failure count:10 • %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=12% Traffic=0% %CONST_DIAG-SP-4-HM_TEST_WARNING: Sup switchover will occur after 10 consecutive failures On Demand Diagnostics Test Failure: • %DIAG-SP-3-TEST_FAIL: Module 5: TestTrafficStress{ID=24} has failed. Error code = 0x1 Scheduled Diagnostics Test Failure: • %DIAG-SP-3-TEST_FAIL: Module 3: TestLoopback{ID=1} has failed. Error code = 0x1 Generic Minor and Major Failure: • %DIAG-SP-3-MINOR: Module 3: Online Diagnostics detected a Minor Error. Please use 'show diagnostic result <target>' to see test results. • %DIAG-SP-3-MAJOR: Module 6: Online Diagnostics detected a Major Error. Please use 'show diagnostic Module 6' to see test results.

23. Reducing Downtime Thru AutomationGOLD Integration With EEM and Call Home • Automates problem diagnosis and information gathering • EEM applets and scripts can initiate GOLD tests • Automates corrective actions and notifications • GOLD events can trigger EEM scripts • Beginning in release 12.2(33)SXH GOLD corrective actions are configured via EEM scripts • Automates result notification • GOLD events are monitored by Call Home diagnostics profile group Configure User Policies Gather Information & Diagnose Known Issues Take Corrective Actions Dispatch & Repair

24. Embedded Event Manager Supports Event Detector for GOLD Core1# show event manager policy register detail Mandatory.go_unusedportlpbk.tcl ::cisco::eem::event_register_gold card all testing_type monitoring test_name TestUnusedPortLoopback action_notify TRUE consecutive_f ailure 10 platform_action 0 queue_priority last # # GOLD TestUnusedPortLoopback Test TCL script # # April 2006, Sifang Li # # Copyright (c) 2005-2007 by cisco Systems, Inc. # All rights reserved. # # # Register for TestUnusedPortLoopback test event # the elements for register the event # card [all | card #] # sub_card [all | sub_card #] # severity_major | severity_minor | severity_normal default : severity_normal # new_failure [true | false] default: dont_care # testing_type [ondemand | schedule | monitoring] # test_name [ test name ] # test_id [ test # ] # consecutive_failure [ consecutive_failure # ] # platform_action [action_flag] # action_flag [ 0 | 1 | 2 ] # queue_priority [ normal | low | high | last] default: normal # #.... • EEM can be used to track and perform corrective actions for GOLD • Beginning in release 12.2(33)SXH all GOLD corrective actions are scripted using EEM

25. Call Home Service Monitors GOLD Status • Automates the notification process • Allows customization via profiles • Severity levels • Who gets notified • Which transport method • Initially supported in IOS 12.2(33)SXH call-home alert-group configuration alert-group diagnostic alert-group environment alert-group inventory alert-group syslog profile "CiscoTAC-1" no active no destination transport-method http destination transport-method email destination address email callhome@cisco.com destination address http https://tools.cisco.com/its/service/oddce/services/DDCEService subscribe-to-alert-group diagnostic severity minor subscribe-to-alert-group environment severity minor subscribe-to-alert-group syslog severity major pattern ".*" subscribe-to-alert-group configuration periodic monthly 8 16:34 subscribe-to-alert-group inventory periodic monthly 8 16:19

26. Generic Online DiagnosticsRecommendations • Bootup diagnostics: • Set level to complete • On demand diagnostics: • Use as a pre-deployment tool: run complete diagnosticsbefore putting hardware into production environment • Use as a troubleshooting tool when suspectinghardware failure • Scheduled diagnostics: • Schedule key diagnostics tests periodically • Schedule all non-disruptive tests periodically • Health-monitoring diagnostics: • Key tests running by default • Enable additional non-disruptive tests for specific functionalities enabled in your network: IPv6, MPLS, NAT

27. Generic Online Diagnostics Summary • Provides a common framework to configure, view and schedule diagnostics across Cisco IOS based switches and routers • GOLD functional tests verify both the data path and control path of the device, can be run during bootup and during runtime • When combined with other features such as Embedded Event Manger and Call Home the MTTR, mean time to repair, can be dramatically lowered via process automation

28. Smart Call Home

29. Catalyst 6500 ManagementSimplified Operation - Smart Call Home Cisco TAC investigates problem and suggests remediation including shipping replacement parts if necessary Customer implements remediation and replaces faulty part (if applicable) Sends message to Cisco TAC with precise information and diagnostics Detects GOLD events and sends to Call Home GOLD runs diags, isolates fault and precise location

30. Customer Notification • Device and Message Reports • Exceptions/Fault Analysis Automated Diagnosis Capability Service Request Tracking System Call Home DB Messages Received: • Diagnostics • Environmental • Syslog • Inventory and Configuration What Is Smart Call Home? Interactive Technical Services Unique Catalyst 6500 Differentiator Customer TAC Internet 3 Secure Transport* 1 2 Call Home *Ensures data protection HTTPS Encryption Certificate-based authentication IOS 12.2(33)SXH

31. 45 min 3.75 hrs 12 hrs 25 hours 29 hours S F TH W T M 1 S 8 7 6 4 15 2 14 13 12 11 22 10 9 21 20 19 P3 Service Request opened Cisco RP team checksIP Multicast configuration Problem narrowed to specific Cat 6500 ports Re-queued to LAN SW team Look into various known issues and bugs on WS-X6548-GE-TX. Find nothing. Request logs from customer Replacement part received (4 –hour replacement coverage) Logs received and analyzed Identify online diagnostics failure for test TestL3VlanMet RMA created 18 29 17 28 16 27 25 29 24 23 28 27 26 25 24 23 30 12 min 12 min 42 min 1.2 hrs 5.5 hrs Minor hardware failure—detected and Service Request automatically generated P3 SR opened due to GOLD failure. Diag. info attached Cisco LAN SW team takes ownership Informs customer of problem and confirms hardware fault RMA createdand part dispatched. Replacement part received (4 –hour replacement coverage) The Smart Call Home Difference Before Minor hardware failure—undetected Customer’s Ops team discovers IP multicast configuration problem After

32. Increased Value Proposition for Cisco Customers Proactive, fast issue resolution • Devices continually monitored with secure, connected service • Real-time alerts for early detection of potential network problems • Automatic, accurate fault diagnosis Higher Network Availability Smart Call Home Less time troubleshooting • Automated Service Request (SR) creation • Detailed diagnostics attached to SR • Routed to correct TAC team Increased Operational Efficiency Fast, web-based access to information • Call Home messages, diagnostics and recommendations • Inventory and configuration for all Call Home devices • Security alerts, Field and End-of-life Notices Fast Access to Information

33. Global Load Balancing Protocol (GLBP)

34. First Hop Routing Protocols • Hot Standby Router Protocol (HSRP) • Cisco informational RFC 2281 ( March 1998) • Patented: US Patent 5,473,599, December 5, 1995 • Virtual Router Redundancy Protocol (VRRP) • IETF Standard RFC 2338 (April 1998) • Now made obsolete by www.ietf.org/rfc/rfc3768.txt • Gateway Load Balancing Protocol (GLBP) • Cisco innovation, load sharing, patent pending

35. Active Standby GLBP cuts useable bandwidth costs in half $648 vs. $1295 6 x T1 = 9.264 Mbps But really only half the links in use, these are idle T1 Costs $1000 $6000 / 9.264 = $648/Mb WAN or MAN Only using 4.632Mbps $1295/Mb Active Active Standby Standby GLBP Business Benefit Suppose a network with dual routers and links, with HSRP GLBP allows use of all available paths

36. Packet rate Packet loss Buffer threshold Packet rate Buffer threshold The Enterprise Premise Edge: Greater Efficiency at Same Cost • With Active/Standby • Single buffer pool, single set of queues • Higher risk of packet loss • With GLBP • Load is shared • More available resources Load balancing improves throughput & reduces potential of packet loss • GLBP improvements over HSRP/VRRP • Simplified provisioning • Improved redundancy model • Superior throughput

37. IP: 10.0.0.254 MAC: 0000.0c12.3456 vIP: 10.0.0.10 vMAC: 0007.b400.0101 IP: 10.0.0.253 MAC: 0000.0C78.9abc vIP: 10.0.0.10 vMAC: 0007.b400.0102 IP: 10.0.0.252 MAC: 0000.0cde.f123 vIP: 10.0.0.10 vMAC: 0007.b400.0103 R1 R2 R3 ARP ARP ARP ARP Reply ARP Reply ARP Reply How GLBP Works R1—AVG; R1, R2, R3 All Forward Traffic GLBP AVG/AVF,SVF GLBP AVF,SVF GLBP AVF,SVF IP: 10.0.0.254 MAC: 0000.0c12.3456 vIP: 10.0.0.10 IP: 10.0.0.253 MAC: 0000.0C78.9abc vIP: 10.0.0.10 IP: 10.0.0.252 MAC: 0000.0cde.f123 vIP: 10.0.0.10 AVG Gateway Routers Clients CL1 CL2 CL3 IP: 10.0.0.1 MAC: aaaa.aaaa.aa01 GW: 10.0.0.10 ARP: IP: 10.0.0.1 MAC: aaaa.aaaa.aa01 GW: 10.0.0.10 ARP: 0007.B400.0101 IP: 10.0.0.2 MAC: aaaa.aaaa.aa02 GW: 10.0.0.10 ARP: 0007.B400.0102 IP: 10.0.0.2 MAC: aaaa.aaaa.aa02 GW: 10.0.0.10 ARP: IP: 10.0.0.3 MAC: aaaa.aaaa.aa03 GW: 10.0.0.10 ARP: 0007.B400.0103 IP: 10.0.0.3 MAC: aaaa.aaaa.aa03 GW: 10.0.0.10 ARP:

38. R1 R2 R3 How GLBP Works R1—AVG; R1, R2, R3 All Forward Traffic GLBP AVG/AVF,SVF GLBP AVF,SVF GLBP AVF,SVF IP: 10.0.0.254 MAC: 0000.0c12.3456 vIP: 10.0.0.10 vMAC: 0007.b400.0101 IP: 10.0.0.253 MAC: 0000.0C78.9abc vIP: 10.0.0.10 vMAC: 0007.b400.0102 IP: 10.0.0.252 MAC: 0000.0cde.f123 vIP: 10.0.0.10 vMAC: 0007.b400.0103 AVG Gateway Routers Clients CL1 CL2 CL3 IP: 10.0.0.1 MAC: aaaa.aaaa.aa01 GW: 10.0.0.10 ARP: 0007.B400.0101 IP: 10.0.0.2 MAC: aaaa.aaaa.aa02 GW: 10.0.0.10 ARP: 0007.B400.0102 IP: 10.0.0.3 MAC: aaaa.aaaa.aa03 GW: 10.0.0.10 ARP: 0007.B400.0103

39. GLBP – Protocol Details • ‘Hello’ messages are exchanged between group members • AVG election by priority • vMAC distribution, learning of VF instances • GLBP will use the following multicast destination for packets sent to all GLBP group members: • 224.0.0.102, UDP port 3222 • Virtual MAC addresses will be of the form: • 0007.b4yy.yyyy • where yy.yyyy equals the lower 24 bits; these bits consist of 6 zero bits, 10 bits that correspond to the GLBP group number, and 8 bits that correspond to the virtual forwarder number • 0007.b400.0102 : last 24 bits = 0000 0000 0000 0001 0000 0010 = GLBP group 1, forwarder 2 • Protocol allows for 1024 groups and 255 forwarders • Number of forwarders are capped at 4 • Hardware restrictions limit actual number of groups and forwarders

40. GLBP Configuration Rules • Load balancing operates on a per-host basis • All connections for a given host will use the same gateway • Maximum of 4 MAC addresses per GLBP Group • Load balancing algorithm, 3 types: • Round-robin • Each virtual forwarder MAC takes turns • Weighted • Directed load determined by advertised weighting factor • Host-dependent • Ensures that each host is always given the same vMAC • If no load balance algorithm is specified, default is round-robin • MD5 authentication security (Releases 12.3(2)T and 12.2(18)S))

41. GLBP Configuration Example ! interface FastEthernet2/0 ip address 10.88.49.1 255.255.255.0 duplex full glbp 1 ip 10.88.49.10 glbp 1 priority 105 glbp 1 authentication text magicword glbp 1 weighting 100 lower 95 glbp 1 weighting track 10 decrement 10 glbp 1 forwarder preempt delay minimum 0

42. Cisco Catalyst 6500 Series and Cisco 7600 Series GLBP Specifics • GLBP “reserves” 4 MAC filter entries • The number of forwarders in the group is limited to 4* • Active Virtual Gateway will ‘allocate’ these to GLBP group members (Virtual Forwarders) • There is a restriction on GLBP group number for the MSFC2/PFC2 – Only a single group may be defined • The single group may be reused on all VLAN • Sup720 supports both plain text & MD5 auth; Sup2 plain text only • HSRP & GLBP can co-exist in Sup720 but not in Sup2 • GLBP Availability: * Note: 1024 group limit is an arbitrary cap, the protocol design actually allows for 4096; as is the forwarder limit of 4 – the design could allow for up to 16. Customers have not requested the additional capacity.