Routing in a Parallel Computer

1. Routing in a Parallel Computer

2. Routing in a Parallel Computer • A network of processors is represented by graph G=(V,E), where |V| = N. • Each processor has unique ID between 1 and N. • Each edge e=(v1,v2) represents a communication link between processors v1 and v2.

3. Routing in a Parallel Computer • All communication proceeds in a sequence of synchronous steps. • Each link ca carry one packet in a step. • During a step processor can send at most one message to each one of his neighbours.

4. Permutation Routing Initially each processor contains packet destined for some processor in the network. • i: processor ID. • vi: packet that processor i contains. • d(i): destination processor of the packet vi. For 1 ≤ i ≤ N, d(i)’s form a permutation of {1,.2,…,N}.

5. Permutation Routing Route: A sequence of edges packet follow from its source to destination. Algorithm’s goal:Find a route for each packet. Note: A packet may occasionally have to wait at an intermediate node because the next edge on its route is "busy" transmitting another packet. We assume that each node contains one queue for each edge leaving the node.

6. Oblivious Algorithm The route followed by vi depends on d(i) alone, and not on d(j) for any j ≠ i. Oblivious routing algorithms are simple to implement. Theorem: For any deterministic oblivious permutation routing algorithm on a network of N nodes each of out-degree d, there is an instance of permutation routing requiring steps.

7. Boolean Hypercube Network • N=2n • n: Hypercube dimensions. • N: Number of nodes. Each node i represented as n-bits binary number: (i0,i1,…,in-1) ϵ {0,1}n There is edge (i,j) iff i and j differs in exactly one bit. Example: (1001, 1101)

8. Boolean Hypercube Network n=3 N=8 001 011 101 111 000 010 100 110

9. Boolean Hypercube Network • Algorithm: sending vi from node i to node σ(i): • Scan the bits of σ(i) from left to right, and compare them with the address of the current location of vi. • Send vi out of the current node along the edge corresponding to the left-most bit in which the current position and σ(i) differ. Example: Sending (1011) to (0000) in 4-dimensional hypercube. Route: (1011) , (0011) , (0001), (0000)

10. Randomized Oblivious Routing Algorithm Phase 1: Pick a random destination σ(i) from {1,...,n}. Packet vi travels to node σ(i). Phase 2: Packet vi travels from σ(i) to its destination d(i). In each phase, each packet uses the bit-fixing strategy to determine its route.

11. Randomized Oblivious Routing Algorithm • Notes: • σ(i) are chosen randomly. So they are not a permutation. • Each node maintains a queue for each outgoing edge, with packets leaving in FIFO order (ties should be broken arbitrarily).

12. Randomized Oblivious Routing Algorithm How many steps elapse before packet vi reaches its destination? • Let us first consider this question for Phase 1: • Let pi be route for vi in Phase 1. • The number of steps taken by vi: • - The length of pi, which is at most n. • - Number of steps it is queued.

13. Randomized Oblivious Routing Algorithm (Lemma 1) • Lemma 1: • View routes in Phase 1 as a directed path. • For two different packets’ routes in Phase 1: once they separate, they do not rejoin.

14. Randomized Oblivious Routing Algorithm (Lemma 1) • Proof: • Each path may be seen as a sequence of binary numbers, representing nodes. • Let’s consider the paths parts starting from the first common node. • Mark: • - v - the last common node. • - v1 – the next node in the first path • - v2 – the next node in the second path

15. Randomized Oblivious Routing Algorithm (Lemma 1) V: c1 b2 V1: b1 c1 V2: b2 c2 b1≠b2 ^ c1≠c2 • The order of updating the bits is left-to-right. • So the bit b1 in the first path and bit b2 in the second path will never change. • And thus the paths will never pass the node with the same ID.

16. Randomized Oblivious Routing Algorithm (Lemma 2) • Lemma 2: • Let the route of vi follow the sequence of edges • pi = (e1,e2,...,ek). • Let S be the set of packets (other than vi) whose routes pass through at least one of {e1,e2,...,ek}. • Then: the delay incurred by vi is at most |S|.

17. Randomized Oblivious Routing Algorithm (Lemma 2) • Proof: • A packet in S is said to leave pi at that time step at which it traverses an edge of pi for the last time. • For each packet v passing pi we define: • lag(v) = t-j  v is ready to follow edge ej at time t. • The lag of vi is initially zero, and the delay incurred by vi is its lag when it traverses ek. We will show that each step at which the lag of vi increases by one can be charged to a distinct member of S. So the time that takes vi to get to its intermediate target is: k+|S| ≤ k+N

18. Randomized Oblivious Routing Algorithm (Lemma 2) • Example: • Next step time is 4 • pi = (e1,e2,e3,e4) v e4 e1 e3 e2 Lag(v) = t – j = 4 – 3 = 1

19. Randomized Oblivious Routing Algorithm (Lemma 2) Claim: If the lag of vi reaches l + 1, some packet in S leaves pi with lag l. Let t’ be the last time step at which any packet in S has lag l. Claim: Some packet of S leaves pi at t’.

20. Randomized Oblivious Routing Algorithm (Lemma 2) When the lag of vi increases from l to l + 1, there must be at least one packet (from S) that wishes to traverse the same edge as vi at that time step, So for each such l there is a packet v such that lag(v)=l.

21. Randomized Oblivious Routing Algorithm (Lemma 2) At time t’ there is a packet v ready to follow edge ej’ at t’ such that t’ – j’ = l. Some packet u in S follows ej’ at t’ (maybe u=v). • Claim: • u leaves pi at t’. • Proof: • Let’s assume it’s not true. •  So u will want to follow the ej’+1 at time t’+1. • lag(u) at time t’+1 is ((t’+1)-(j’+1)) = l. • In contradiction that t’ is the last time when any packet in S has lag l.

22. Randomized Oblivious Routing Algorithm (Lemma 2) Proof summary: When vi is delayed, one packet in S leaves pi. According to Lemma 1, once the paths are separated, they never rejoin.  After |S| delays no packet except vi travels thru pi.  vi may be delayed up to |S| steps.  It takes vi up to k + |S| steps to get to it’s intermediate target.

23. Randomized Oblivious Routing Algorithm • Random variables Hij: • Hij = 1 , if pi and pj share at least one edge • Hij = 0 , otherwise The intermediate nodes for each packet are chosen independently. So Hij for some i are independent. By lemma 2, delay of package vi is at most:

24. Randomized Oblivious Routing Algorithm

25. Randomized Oblivious Routing Algorithm Random variables T(e): The number of routes that pass through edge e. For any route pi=(e1,e2,…,ek):

26. Randomized Oblivious Routing Algorithm Lemma:

27. Randomized Oblivious Routing Algorithm

28. Randomized Oblivious Routing Algorithm

29. Randomized Oblivious Routing Algorithm The probability for package to wait in Phase 1 for more than 6n steps is less than 2-6n. The probability for one of the packages to wait in Phase 1 for more than 6n steps is less than N * 2-6n =2n * 2-6n = 2-5n. Add the length of the longest possible path and we get that the probability to finish Phase 1 in more than 7n steps is less than 2-5n.

30. Randomized Oblivious Routing Algorithm Phase 2: Phase 2 can be seen as reflection of Phase 1. Paths of Phase 2 can intersect with paths of Phase 1. In order to prevent this we wait 7n steps to be sure all packets got to their intermediate destination, before starting Phase 2.

31. Randomized Oblivious Routing Algorithm The probability for the process not to finish in 2 * 7n steps is less than 2 * 2-5n = 2-4n. With probability at least 1 - (1/N), every packet reaches its destination in 14n or fewer steps.

32. Summary The deterministic algorithm takes: steps. The randomized 14n steps. The probability algorithm is exponentially better than the deterministic one.