Interconnection Network Topology Design Trade-offs

1. Interconnection Network Topology Design Trade-offs

2. Organizational Structure • Processors • datapath + control logic • control logic determined by examining register transfers in the datapath • Networks • links • switches • network interfaces

3. Link Design/Engineering Space • Cable of one or more wires/fibers with connectors at the ends attached to switches or interfaces Synchronous: - source & dest on same clock Narrow: - control, data and timing multiplexed on wire Short: - single logical value at a time Long: - stream of logical values at a time Asynchronous: - source encodes clock in signal Wide: - control, data and timing on separate wires

4. Example: Cray MPPs • T3D: Short, Wide, Synchronous (300 MB/s) • 24 bits • 16 data, 4 control, 4 reverse direction flow control • single 150 MHz clock (including processor) • flit = phit = 16 bits • two control bits identify flit type (idle and framing) • no-info, routing tag, packet, end-of-packet • T3E: long, wide, asynchronous (500 MB/s) • 14 bits, 375 MHz, LVDS • flit = 5 phits = 70 bits • 64 bits data + 6 control • switches operate at 75 MHz • framed into 1-word and 8-word read/write request packets

5. Switches

6. Switch Components • Output ports • transmitter (typically drives clock and data) • Input ports • synchronizer aligns data signal with local clock domain • essentially FIFO buffer • Crossbar • connects each input to any output • degree limited by area or pinout • Buffering • Control logic • complexity depends on routing logic and scheduling algorithm • determine output port for each incoming packet • arbitrate among inputs directed at same output

7. Interconnection Topologies Topology [Regular] [Irregular] [Static] [Dynamic] [Hypercube] [Single- Stage] [Multistage] [Crossbar] [....] [Two- Dimensional] [Three- Dimensional] [....] [One- Dimensional] [One- Sided] [Two- Sided]

8. Static Connection Topologies • Mesh and Torus • Illiac IV, MPP, DAP, CM-2, Paragon • k-dimensional mesh N=nk, d=2k, D=k(n-1) • wraparound variation - Illiac IV • Torus n x n binary torus, d = 4, D = 2ën/2û • Hypercubes • iPSC, nCube, CM-2 • N = 2n, d = n, D = n • poor scalability, difficulty in packaging higher-dimensional hypercubes

9. Dynamic Interconnection Networks • Bus-based networks • Crossbar networks • Single Stage Networks • Shuffle-exchange • N input and N output • Crossbar • Recirculating networks • Multi-stage Networks • more than one stage of switching elements • switching box: straight, exchange, upper broadcast, lower broadcast • network topology and control structure

10. Dynamic Interconnection Networks • Two-sided MIN • connecting an arbitrary input to an arbitrary output • blocking, rearrangeable, nonblocking networks • blocking networks • Data manipulator, Omega, Flip, n-cube, Baseline • rearrangeable networks • Benes network • nonblocking networks • Clos, Crossbar

11. Interconnection Topologies • Logical Properties: • distance, degree • Physcial properties • length, width • Fully connected network • diameter = 1 • degree = N • cost? • bus => O(N), but BW is O(1) - actually worse • crossbar => O(N2) for BW O(N) • VLSI technology determines switch degree

12. Linear Arrays and Rings • Linear Array • Diameter? N-1 • Average Distance? 2/3N • Bisection bandwidth? 1 • Route A -> B given by relative address R = B-A • Space O(N) • Torus? Or Ring • Examples: FDDI, SCI, FiberChannel Arbitrated Loop, KSR1

13. Multidimensional Meshes and Tori • d-dimensional array • N = kd-1 X ...X kO nodes • described by d-vector of coordinates (id-1, ..., iO) • d-dimensional k-ary mesh: N = kd • k = dÖN • described by d-vector of radix k coordinate • d-dimensional k-ary torus (or k-ary d-cube)? 3D Cube 2D Grid

14. Properties • Routing • relative distance: R = (b d-1 - a d-1, ... , b0 - a0 ) • traverse ri = b i - a ihopsin each dimension • dimension-order routing • Average Distance Wire Length? • d x 2k/3 for mesh • dk/2 for cube • Degree? • Bisection bandwidth? Partitioning? • k d-1 bidirectional links • Physical layout? • 2D in O(N) space Short wires • higher dimension?

15. Real World 2D mesh • 1824 node Paragon: 16 x 114 array • a single cabinet: 16 X 4 array

16. Embeddings in two dimensions • Embed multiple logical dimension in onephysical dimension using long wires 6 x 3 x 2

17. Trees • Diameter and ave distance logarithmic • k-ary tree, height d = logk N • address specified d-vector of radix k coordinates describing path down from root • Fixed degree • Route up to common ancestor and down • R = B xor A • let i be position of most significant 1 in R, route up i+1 levels • down in direction given by low i+1 bits of B • H-tree space is O(N) with O(ÖN) long wires • Bisection BW?

18. Fat-Trees • Fatter links (really more of them) as you go up, so bisection BW scales with N

19. Butterflies • Tree with lots of roots! • N log N switches (actually N/2 x logN) • Exactly one route from any source to any dest • R = A xor B, at level i use ‘straight’ edge if ri=0, otherwise cross edge • Bisection N/2 vs N (d-1)/d (d-dimensional mesh) vs 1 (tree) building block 16 node butterfly

20. Benes network and Fat Tree • Back-to-back butterfly can route all permutations • off line • What if you just pick a random mid point?

21. Hypercubes • Also called binary n-cubes. # of nodes = N = 2n. • O(logN) Hops • Good bisection BW • Complexity • Out degree is n = logN 0-D 1-D 2-D 3-D 4-D 5-D !

22. Relationship BttrFlies to Hypercubes • Wiring is isomorphic • Except that Butterfly always takes log n steps

23. Toplology Summary • All have some “bad permutations” • many popular permutations are very bad for meshs (transpose) • randomness in wiring or routing makes it hard to find a bad one! Topology Degree Diameter Ave Dist Bisection D (D ave) @ P=1024 1D Array 2 N-1 N / 3 1 huge 1D Ring 2 N/2 N/4 2 2D Mesh 4 2 (N1/2 - 1) 2/3 N1/2 N1/2 63 (21) 2D Torus 4 N1/2 1/2 N1/2 2N1/2 32 (16) k-ary n-cube 2n n(k-1) n(k-1)/2 2kn-1 27 (13.5) @n=3 Hypercube n=log N n n/2 N/2 10 (5)

24. Wire Efficient CommunicationNetworks for Multicomputers • What makes a network efficient? • Efficient use of the limiting resources • Limiting Factors • switches and pins were only considered the limiting factors • Wires are limiting factors because of power and delay as well as density • At the board level as well as at the chip level, the system interconnection is limited by wire density • Most of the power dissipated in the networks is CV2fpower to used to drive wires. • Most of the delay is propagation delay over wires or RC delay in driving wires

25. In the 3D world • For n nodes, bisection area is O(n2/3 ) • For large n, bisection bandwidth is limited to O(n2/3 ) • Bill Dally, IEEE TPDS, [Dal90a] • For fixed bisection bandwidth, low-dimensional k-ary d-cubes are better (otherwise higher is better) • i.e., a few short fat wires are better than many long thin wires • What about many long fat wires?

26. The Design Objective of the Network • To minimize latency and maximize throughput • Latency T(l,L) :the average time required to deliver a message • Each node injects messages with average length L into the network at an average rate of l bits per cycle. • Three independent variables: topology, routing, and flow control • Topology Indirect Networks (k-ary d-flys: radix k and dimension d) • No of processing nodes: N = kd BI = N/2 : high bisection width BWI = Nw/2 din = dout = k d = 2k : low degree D = d+1 : low diameter 2-ary 3-fly

27. Wire Efficient Topology • Indirect Networks • high bisection width, low degree, low diameter, long wires, symmetry • the bisection width B = N/2 does not reflect the actual maximum wire density for this class of networks: vertical partition (N wires) more accurately reflects the wiring problems • wire area O(N2) : plane mapping - expensive • N = kd. As one varies k and d with the number of processing nodes, N, and BW fixed. • the degree and diameter are directly controlled. • the channel width remains fixed at w = BW/B=2BW/N. • B is independent of the choice of k and d. • disadvantage: it prevents the designer from trading off the bandwidth of a channel against the diameter of the network.

28. Wire Efficient Topology • Direct Networks (k-ary d-cubes) • BD = 2N/k • BWD = 2Nw/k • din = dout = d • d = 2d • D = dk/2 • For small d • a low and controllable bisection width (N=kd) • low degree • high diameter • short wires (d£ 3) • wiring complexity O(N) BI = N/2 : high bisection width BWI = Nw/2 din = dout = k d = 2k : low degree D = d+1 : low diameter

29. How Many Dimensions? • d = 2 or d = 3 • Short wires, easy to build • Many hops, low bisection bandwidth • Requires traffic locality • d ³4 • Harder to build, more wires, longer average length • Fewer hops, better bisection bandwidth • Can handle non-local traffic • k-ary d-cubes provide a consistent framework for comparison • N = kd • scale dimension (d) or nodes per dimension (k) • assume cut-through

30. Traditional Scaling: Unloaded Latency(N) • Assumes equal channel width • independent of node count or dimension • dominated by average distance Unit routing delay (D = 1) w = 1

31. Real Machines • Wide links, smaller routing delay • Tremendous variation

32. Average Distance • but, equal channel width is not equal cost! • Higher dimension => more channels ave dist = d (k-1)/2