LOAD BALANCING SWITCH

1. Final presentation for project By: Maxim Fudim Oleg Schtofenmaher Supervisor: Walter Isaschar • Spring 2008 ( Part B) LOAD BALANCING SWITCH

2. General overview • Software solutions for real-time are too slow • Power dissipation limits work frequencies • Greater computing power needed • H/W accelerators can improve S/W processes • Multi-core, multi-threaded systems are the future

3. Project Goals • Multiprocessor environment for parallel processing of vectors data stream • Maximal Throughput • Configurable hardware • Expandable design • Statistics report

4. System specifications • SW over transparent HW • Interface over PCI • 1 Mbit/sec input stream • Vectors of 8 ÷ 1024 chunks • Variable number of processors • System spreads over multiple FPGAs

5. Problem • How to manage Data stream? • How to manage multiple parallel units? • How to achieve full and effective utilization of resources?

6. Solution (Top Level) • Board Level Load Balancing Switch • One system input and output to PCI • Distribute vectors among classes • Local buffers for chip data

7. Solution (Chip Level) • Chip Level Load Balancing Switch • Converting shared resources to “personal” work space. • Cluster ‘s organized VPUs • Monitoring for each unit’s load • Smart arbitration • Flexible and easy configuration

8. Solution - Tree Distribution Switch SW/HW interface Class of Service Distribution LBS Arbitration LBS Arbitration LBS Arbitration Clusters of VPUs Clusters of VPUs Clusters of VPUs Clusters of VPUs Clusters of VPUs Clusters of VPUs Clusters of VPUs Clusters of VPUs Clusters of VPUs

9. Three level Architecture • Provide level for packets management ( Classes ) • Type, Size, Priority of Data • Provide level for organizing various processing units ( Clusters ) • Speed , Quantity, Resources of Processors • Provide level for fine tuning ( VPUs ) • Algorithm, HW accelerating

10. Implementation

11. Board Level • Multi chip system • Local FIFOs for every chip/class • Classifier for packet management • SW configurable controls • Input and Controls over Main Bus • Output via streamed neighbored busses

12. Board Overview

13. Busses Description

14. Board Level diagram Stratix II 180 Stratix II 180 Stratix II 180 Stratix II 180 Ring Bus Ring Bus LBS1 LBS2 LBS3 LBS4 Classifier MainBus : Data In and Controls DDR2 DDR2 DDR2 DDR2 DDR2 DDR2 DDR2 DDR2 PROCStar II Input vectors Output reports Per LBS registers PCI Bus S/W emulator or H/W DSP system

15. Single Chip diagram Load Balancing Switch (LBS) Data and Controls NIOS VPU NIOS VPU NIOS VPU NIOS VPU Stratix II FPGA Bus Control Block Left Bus Muxed Reports Right Bus Reports DDR2 A FIFO IN DDR2 B FIFO OUT Main Bus Input Vectors

16. PCI-System Interfaces Software - Hardware Interface: • Input and Output MultiFIFO PCI data bus • MultiFIFO status LBS 1-4 Interface: • 2x32-bit general read purpose registers • 2x32-bit general write purpose registers • 8-bit information register • Software reset signal

17. PCI-System Interfaces Classifier: • Global Configuration Register (32 bit) • Global Info Register (32 bit) • Global In Count Register (32 bit) • Global Out Count Register (32 bit) • Global Active Time Register (32 bit) • Global Software reset signal

18. Board Level Description Classifier (board level): • Distributes data from Input PCI to Local FIFOs • Handles demands from Local Output Masters • Synchronize data and controls • Configurable arbitration between LBS classes • Configurable statistics gathering • Timeout mechanism

19. Board Level Description Busses Control Block (on every chip): • Parametric pins numbering • Main /Ring Busses routing • Data sampling • FIFO management • Local Grant controls • Local Output FIFO master

20. Main Bus Interfaces • Input Data & Control Interface: • Input data bus to Local FIFOs • ACK from Local FIFOs • REQ to Local FIFOs • Statistics REQ

21. Main Bus Interfaces Output Controls Interface: • Demand from Local FIFO Masters • Output Grant • ACK from PCI FIFO • End of vector from PCI FIFO Master

22. RING Bus Interfaces • Output Data Interface: • Output data bus from Local FIFOs • Data Valid from Local Masters • End of output Vector from Local Masters • Statistics Data • Statistics Valid

23. Chip Level • Local FIFO for inputs/outputs • Internal clusters configuration • Arbitration, priorities • Statistics, Synchronization

24. Single FPGA Top Diagram NIOScluster NIOScluster NIOScluster NIOScluster DDR2 Controls Bank A NIOScluster NIOScluster NIOScluster NIOScluster Bus Control Block Load Balancing Switch (LBS) I/O – LBS Control Block Data flow NIOScluster NIOScluster NIOScluster NIOScluster DDR2 Controls Bank B NIOScluster NIOScluster NIOScluster NIOScluster LBS 1-4 Stratix II 180 FPGA

25. Input System Interface LBS Input Interface: • 64 bit data bus from Input MultiFIFO • Read request and ack. Signals • MultiFIFO status flags • SW/HW input signals

26. Output System Interface LBS Output interface: • 64 bit data bus to Output MultiFIFO • Write request and ack. Signals • MultiFIFO status flags • SW/HW input signals

27. Data Packet Format …… Header Tail Data 1 to N of 32-bit Words Header : SW/HW Control 1-bit Unused Nios Number Data Length N Vector ID/Command Type Type 1-bit (Data/Command) Version 4-bit 8-bit 16-bit 32-bit Tail : Sync Data

28. LBS Top Level View FIFO Input Port Input data bus Cluster Arbiter Cluster Arbiter Cluster Arbiter PCI Input Reader NIOS II System NIOS II System NIOS II System Control Main Controller unit Statistics Reporter Control Control and Status Output Writer Control FIFOOutputPort Muxed output data bus Stratix II FPGA

29. Organization of VPU’s(Vector Processing Units) • NIOS VPUs joined into the clusters • Constant number of Clusters • Parametric number of NIOS VPU’s in cluster • Parametric control logic • Variable configuration of NIOS • Different Priority for different clusters

30. NIOS Input Interface Hardware: • 64-bit input data bus – from LBS • 10 bit data slices counter – from LBS • Write request signal – from LBS • Chip select signal – from LBS • NIOS ready signal – from NIOS • Data ready signal – from LBS

31. NIOS Output Interface Hardware: • 64 bit output data bus – from NIOS • 7 bit data slices counter – from LBS • Read request signal – from LBS • Chip select signal – from LBS • Output ready signal – from NIOS • Output taken signal – from LBS

32. Twin VPU SystemInput / Output waveform

33. FIFO Input Port Input data bus LBS Units DescriptionInput Reader Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Reading data from input FIFO • Writing data to selected cluster • Providing header control bits for main controller • Synchronization checks • Vector length counter • Input Time stamp Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

34. FIFO Input Port Input data bus LBS Units Description Sync Flusher Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus Cluster Arbiter Cluster Arbiter Cluster Arbiter • Flush data on Input error • Look for Sync Tail • Parametric number of recovery tries • Failure signal to Error Reporter NIOS II System NIOS II System NIOS II System

35. Input Reader Diagram

36. LBS Units DescriptionInput Controller - FSM

37. FIFO Input Port Input data bus LBS Units DescriptionOutput Writer Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Reading data from selected cluster • Writing data to output FIFO • Vector length counter • Output Time Stamp Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

38. Output Writer Diagram

39. LBS Units DescriptionOutput Controller - FSM

40. FIFO Input Port Input data bus LBS Units DescriptionMain Controller Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Enabling input and output units • Selecting control source (S/W or H/W) • Monitoring clusters’ load via status buses • Selecting clusters for input/output operations • Data validity indication Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

41. Main ControllerStatus Decoders

42. LBS Units DescriptionMC Status Alghoritm • Status input and output independent decoders • Static Priority • Dynamic Load • Parametric Aging mechanism • Round Robin in same priority group

43. LBS Units DescriptionMC Status Alghoritm Status input Static Priority/ Aging mapping Dynamic port mapping RR on Active ports Next port 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 1 1 0 1 1 1 1 1 0 0 0 1 14|3 15|0 13|0 1|7 4|7 2|1 3|13 4|12 0|0 3|7 15 13 15 14 13 14 3 1 4 1 0 2 3 3 0 2 4 0 0 1 1 1 ... ... ... ... 2 0 2

44. Decoding Flow

45. FIFO Input Port Input data bus LBS Units DescriptionStatistics Reporter Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Monitoring system activity • Error reporting for software • Counting processed vectors • Throughput = Vectors served / Time of service Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

46. FIFO Input Port Input data bus LBS Units DescriptionClusters Load Reporter Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Monitoring clusters activity • Per VPU active/free status • Sending Status by request from Classifier Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

47. FIFO Input Port Input data bus LBS Units DescriptionError Reporting Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Input Reader error • Recovery Synchronize Failure • Packets Drops • Local Output Master’s Error • LBS’s activity Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

48. FIFO Input Port Input data bus LBS Units DescriptionCluster Entity Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Cluster/VPUs parametric enabling • Cluster/VPUs status reporters • VPUs Flow controllers • Watchdogs • NIOS Systems Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

49. FIFO Input Port Input data bus LBS Units Description Cluster Configs Input Reader Main Controller unit Statistics Reporter Control and Status Output Writer FIFOOutputPort Muxed output data bus • Define quantity of VPUs ports • Define type of VPUs in cluster • Automatic creation of per VPU control logic • Parametric arbiter for input/output data Cluster Arbiter Cluster Arbiter Cluster Arbiter NIOS II System NIOS II System NIOS II System

50. LBS Units DescriptionPer Nios Structure