Efficient Link Capacity and QoS Design for Wormhole Network-on-Chip

1. Efficient Link Capacity and QoS Design for Wormhole Network-on-Chip Zvika Guz, Isask’har Walter, Evgeny Bolotin, Israel Cidon, Ran Ginosar and Avinoam Kolodny Technion, Israel Institute of Technology

2. Problem Essence • How much capacity [bits/sec] should be assigned to each link? • All flows must meet delay requirements • Minimize total resources R R R R R R R R R R R R R R R R R NoC Capacity Allocation

3. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole based NoC • The problem of link capacity allocation • Solution: • Wormhole delay model • Capacity allocation algorithm • Design examples • Summary NoC Capacity Allocation

4. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole based NoC • The problem of link capacity allocation • Solution: • Wormhole delay model • capacity allocation algorithm • Design examples • Summary NoC Capacity Allocation

5. Wormhole Switching • Suits on chip interconnect • Small number of buffers • Low latency • Virtual Channels • interleaving packets on the same link IP2 Interface Interface IP1 NoC Capacity Allocation

6. Wormhole Switching IP3 • Suits on chip interconnect • Small number of buffers • Low latency • Virtual Channels • interleaving packets on the same link IP2 Interface Interface Interface IP1 NoC Capacity Allocation

7. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole basedNoC • The problem of link capacity allocation • Solution: • Wormhole delay model • Capacity allocation algorithm • Design examples • Summary NoC Capacity Allocation

8. NoC Design Flow R R R R R R R R R R R R R R R R R R R R R R R R R R Define inter-module traffic Module Module Module Module Module R R R Module Module Place modules R R R R R Module Module Module Module Module R R R R Module Module Module Allocate link capacities R R Module Module Verify QoS and cost NoC Capacity Allocation

9. NoC Design Flow R R R R R R R R Module Module Module Module Module Module Module Module R R Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module Module Module R R R R R R R R R Module Module Module Module Module Module Module R R Module Module R R Module Module R R R R R R R R R Module R R R R Module Module Module Module Module Module Module Module Module Module R R R R R R Module Module Module Module R R R R R R R R R Module Module Module Module Module Module Module Module Module R R R R Module Module R R Module Module • Too low capacity results in poor QoS • Too high capacity wastes power/area Define inter-module traffic Place modules Allocate link capacities Verify QoS and cost NoC Capacity Allocation

10. Capacity Allocation Problem • Simulation takes too long • a simulation based solution is not scalable • If no simulations are used: • How to extract flows’ delays? • How to reassign capacity? • Our solution: • Analytical model to forecast QoS • Capacity allocation algorithm that exploit the model NoC Capacity Allocation

11. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole based NoC • The problem of link capacity allocation • Solution: • Wormhole delay model • Capacity allocation algorithm • Design examples • Summary NoC Capacity Allocation

12. Delay Analysis • Approximate per-flow latencies • Given: • Network topology • Link capacities • Communication demands R R R R s1 R R R R R R R s2 d2 R R R R R R d1 NoC Capacity Allocation

13. Why Previous Models Do Not Apply? • Because they assume: • Symmetrical communication demands • No virtual channels • Identical link capacity! • Generally, they calculate the delay of an“average flow” • A per-flow analysis is needed NoC Capacity Allocation

14. Wormhole Delay Analysis IP2 • The delivery resembles a pipeline pass • Packet transmission can be divided into two separated phases: • Path acquisition • Packet delivery • We focus on packet delivery phase Interface Interface IP1 NoC Capacity Allocation

15. Packet Delivery Time IP2 • Packet delivery time is dominated by the slowest link • Transmission rate • Link sharing Low-capacity link Interface Interface IP1 NoC Capacity Allocation

16. Packet Delivery Time IP3 IP2 • Packet delivery time is dominated by the slowest link • Transmission rate • Link sharing Interface Interface Interface IP1 NoC Capacity Allocation

17. Analysis Basics t t • Determines the flow’s effective bandwidth • Per link • Account for interleaving NoC Capacity Allocation

18. Single Hop Flow, no Sharing t • - mean time to deliver a flit of flow i over link j [sec] • - capacity of link j [bits per sec] • - flit length [bits/flit] • - total flit injection rate of all flows sharing link j except for flow i [flits/sec] NoC Capacity Allocation

19. Single Hop Flow, with Sharing t Bandwidth used by other flows on link j • - mean time to deliver a flit of flow i over link j [sec] • - capacity of link j [bits per sec] • - flit length [bits/flit] • - total flit injection rate of all flows sharing link j except for flow i [flits/sec] t NoC Capacity Allocation

20. The Convoy Effect Account for all subsequent hops Link Load Basic delay weighted by distance • Consider inter-link dependencies • Wormhole backpressure • Traffic jams down the road NoC Capacity Allocation

21. Total Packet Transmission Time Packet size[flits/packet] = Account for weakest link • Weakest link dominates packet delivery time • - mean packet latency for flow i [sec] • - mean packet latency for flow i [sec] NoC Capacity Allocation

22. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole based NoC • The problem of link capacity allocation • Solution: • Wormhole delay model • Capacity allocation algorithm • Design examples • Summary NoC Capacity Allocation

23. Capacity Allocation Algorithm • Greedy, iterative algorithm For each src-dst pair: • Use delay model to identify most sensitive link • Increase its capacity • Repeat until delay requirements are met NoC Capacity Allocation

24. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole based NoC • The problem of link capacity allocation • Solution: • Wormhole delay model • Capacity allocation algorithm • Design examples • Summary NoC Capacity Allocation

25. Capacity Allocation – Example#1 Uniform traffic with identical requirements Uniform allocation: 74.4Gbit/sec 03 02 13 01 12 23 00 11 22 33 10 21 32 20 31 30 • Capacity allocation algorithm: 69Gbit/sec Total capacity reduced by 7% Before optimization After optimization

26. A SoC-like system Heterogeneous traffic demands and delay requirements Uniform allocation: 41.8Gbit/sec Capacity Allocation – Example#2 • Capacity allocation algorithm: 28.7Gbit/sec Total capacity reduced by 30% 03 02 Before optimization 13 01 12 After optimization 23 00 11 22 10 21 20

27. Outline R R R R Module Module Module Module R R Module Module R R R R R Module Module Module Module Module R R R R Module Module R R Module Module • Wormhole based NoC • The problem of link capacity allocation • Solution: • Wormhole delay model • Capacity allocation algorithm • Design Examples • Summary NoC Capacity Allocation

28. Summary • SoCs need non uniform link capacities • Capacity allocation • Wormhole delay analysis • Heterogeneous link capacities • Heterogeneous communication demands • Multiple VCs • Greedy allocation algorithm • Design examples • NoC cost considerably reduced NoC Capacity Allocation

29. Questions? QNoC Research Group QNoC Research Group NoC Capacity Allocation

30. Backup NoC Capacity Allocation

31. QNoC Architecture Grid topology Packet-switched Wormhole switching Fixed path XY routing Heterogeneous link capacities Quality-of-Service Router Link R R R R R Module Module Module Module Module R R R R R Module Module Module Module Module R R R R R Module Module Module Module Module R R R R R Module Module Module Module Module R R R R R Module Module Module Module Module E. Bolotin, I. Cidon, R. Ginosar, A. Kolodny, “QoS Architecture and Design Process for Cost-Effective Network on Chip”, Journal of Systems Architecture, 2004 NoC Capacity Allocation

32. Analysis Validation Analysis and Simulation vs. Load Normalized Delay Utilization • Analytical model was validated using simulations • Different link capacities • Different communicationdemands NoC Capacity Allocation

33. Slack Elimination Packet Delay Slack Slack [%] Flow NoC Capacity Allocation