Fibre Channel - Topologies & Protocols

1. Fibre Channel - Topologies & Protocols

2. Agenda • Why Fibre • Fibre-Speak • Fibre Pipes • Topologies • Protocols

3. And most Brits have troublewith words longer thanP-U-B and P-I-N-T. Question • Why is Fibre in Fibre Channel always misspelled F-I-B-R-E instead of F-I-B-E-R? • Answer: It’s the Britishspelling of the word.

4. Why Fibre

5. Why Fibre Channel? • Speed • 100 MB/s now • 200 MB/s early next year • 400 MB/s in the future • Distance • 30 meters with copper • 100’s of meters with today’s multi-mode fibre • 1000’s of meters with tomorrow’s single mode fibre • Flexibility • Point-to-point • Loops • Fabrics • It’s the industry standard interconnect • Clusters and SAN’s now • And an important disk interface

6. What’s a Cluster? • Group of independent systems that • Function as a single system • Appear to users as a single system • And are managed as a single system • Clusters are “virtual servers” Fibre Channel

7. What’s a SAN? • Dedicated Network for Storage Devices • RAID Arrays, Tape and Optical libraries • Storage management software, e.g Back-up, archiving,, … • “Network Behind the Servers” LAN SAN Fibre Channel

8. What’s a Fibre Disk? • SCSI Disk • Single SCSI interface • Fibre Disk • Dual Fibre Channel interfaces • If one fibre loop fails, you can still access a fibre disk SCSI Disks Fibre Disks

9. And It’s A Key Mylex Strategy • SF and FL Controllers • Fibre to the server, cluster or SAN • SCSI to the disks • FF Controller • Fibre to the server, cluster or SAN • Fibre to the disks • Fibre Apple, Lion and Lambersomethingorother • PCI to the server • Fibre to the disks • Fibre to another Fibre Apple or Lion (cluster configuration)

10. Fibre-Speak

11. Fibre-Speak • Fibre Node • Any device on a fibre loop or fabric with a fibre port address • Servers, Array controllers, Tape Libraries, ... • Fibre Networking Devices • Devices used to interconnect nodes in loop or fabrics • Fibre Hubs and Fibre Switches • Fibre Loop • Topology where nodes are configured in a loop like Token Ring • Fibre networking device -- Hub(s) • Fibre Fabric • Topology where nodes are configured in a star or mesh (“the cloud”) • Fibre networking device -- Switch(es) • And it could have hub(s) hanging off switches

12. More Fibre Speak • GBIC • Gigabit Interface Converter • Converts optical signal to copper signal (and vice versa) • Can plug into any fibre port • DB-9 and HSSDC • Fibre connectors that provide a fibre port • External product development kit (SF-DEVKIT) have HSSDC’s • LIP • Loop initialization process • Similar to a SCSI bus reset • LRC • Loop redundancy circuit • Allows fibre loops to bypass non-operational fibre nodes

13. Fibre Pipes

14. Gigabit Transmission • SCSI • Ultra 40 MB / s • Ultra/2 LVD 80 MB / s -- Now • Ultra/3 LVD 160 MB / s -- Late ‘99 • Fibre Channel • 1st Generation Gb Fibre 100 MB/s now • 2nd Generation 2 Gb Fibre 200 MB/s 2000 • 3rd Generation 4 Gb Fibre 400 MB/s 2001

15. Versus SCSI which is an 8 or 16 bit wide parallel bus • 8 or 16 bits at a time across a bus 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 0 0 1 1 0 Serial Transmission • Fibre is a serial technology • One bit at a time through a pipe 001001110101001 • Serial pipes can be designed to go faster over greater distances

16. Fibre Cables • Fibre cables have two pipes • One to move bits in each direction • Also called a “link” • Two unidirectional “fibres” transmitting in opposite directions • “fibres” -- copper or optical 001001110101001 001001110101001

17. Fibre Channel Pipes • Fibre Channel defines copper and optical pipes • Copper Cables • Electrons represent bits • Cheaper • Distance limited -- 30 meters • Optical Fibre Cables • Light pulses represent bits • Pulses are generated by a laser or a LED • And detected at the other end by a transceiver • More expensive • Distances up to thousands of meters • Depends on type of optical pipe, transmission speed, ...

18. Fibre Channel Glass Pipes • Multi-Mode Optical Fibre -- Now • Pipe is large enough to simultaneously transmit multiple streams of light • Comes in two diameters • 50 microns -- de facto standard • 62.5 microns • Single Mode Optical Fibre -- Future • Pipe is only wide enough to transmit a single steam of light • 9 microns

19. Relative size of a one micron particle Average brain size of East Coast Sales Guy Question • How big is a micron?

20. Speed / Distance of FC Pipes • Copper-Local Area Wiring • Multi-Mode Building Wiring • Single Mode -Campus Wiring Transmission Speed 100 MB / s 200 MB / s 400 MB / s Copper 30 m 62.5 Micron Multi-Mode 175 m 150 m 90 m 50 Micron Multi-Mode 500 m 300 m 150 m 9 Micron Single Mode 10 km 2 km 2 km

21. Management Question • How big is a meter? • Hint: 1,000 Microns = Meter • Answer: Enough space to put 4,000 east coast brains side by side.

22. Topologies

23. Topologies • Physical Model -- Topologies • How the servers, arrays, hub and switches are physically connected • Communications Model • How nodes talk to each other and it’s always “point-to-point” • At any point in time, one node talks to one other node • Fibre Ports • N-Port (node in a fabric topology) • F-Port (switch port in a fabric topology) • NL-Port (node in a loop topology • FL-Port (hub port in a loop topology)FC Node = server or an arrayFC Ports have world-wide addresses like Ethernet portsSF, FL and FF have two FC Ports (addresses stored in controllers)

24. Point-to-Point Arbitrated Loop (FC-AL) Fabric Three Fibre Topologies

25. Fibre Hubs • Networking Device for Connecting Fibre Nodes in a Loop • Simplifies cabling • Connects fibre nodes with different fibre cable types • Can plug copper and different types of optical into the same hub • Improves cluster or SAN availability • Allows fibre nodes to be hot plugged and removed • Bandwidth in a Loop is Constant (100 MB/s Today) • As servers are added to a loop; less bandwidth per server • Cheaper Per Port than Fibre Switches

26. FC Loop w/ Four Port Fibre Hubs Hub SF SF

27. Server Failure - HUB LRC Heals Loop LRC Isolates Dead Server X Hub SF SF

28. SF Failure -- HUB LRC Heals Loop Hub X SF SF

29. Fibre Switches • Networking Device for Connecting Fibre Nodes in a Fabric • Does everything a hub does • Same benefits PLUS • Bandwidth in a Fabric is Scaleable • Add more servers to a loop • Bandwidth per server is constant • But bandwidth for the fabric increases with each added node • Expensive Per Port Compared to Fibre Hubs

30. FC Fabric w/ 4 Port Fibre Switch - 200 MB/s Pairs of Nodes Communicate in Loops and Fabrics S0 S1 Switch But They Communicate Simultaneously in Fabrics SF 0 SF 1

31. FC Fabric w/ 4 Port Fibre Switch - 200 MB/s S0 S1 Switch Any node can talk to any other node SF 0 SF 1

32. FC Fabric w/ 8 Port Fibre Switch - 400 MB/s S0 S1 S0 S1 Switch SF 0 SF 1 SF 0 SF 1

33. Fibre Protocols

34. Fibre Channel Protocol • Fibre Channel Spec Defines • Physical aspects • Nodes, ports, links (cables), … • And the communications model • Frame -- carries the payload (chunks of data) • Sequence -- set of related frames (for flow control) • Exchange -- set of related sequences (sets up the conversation) • Fibre Protocol is a “Low Level” Protocol • And is used to transport other protocols • It transports the SCSI protocol to talk to disks (SCSI over FC) • It can transport the IP protocol for array management • Plan to port the GAM server to external controllers • And talk to it from WAM clients (IP over FC)

35. Confidentlyhead out • Example: Ron tells you, Get the order today! • Dazzle them with product knowledge • Close withfinesse • You grab yoursales trainingpresentations • Ron reactspositively Last Question • What’s a protocol?

36. You can findyour salestraining presentation • You come back withnothing morethan a goodstory • You head outwondering howto pull it off • Ron gets akick out of your story • Decide to try begging • And then stops laughing Or