Straight Line Code

1. Straight Line Code xi = a xi = xj + xk xi = xj xk No loop No index as variables No if statement Arithmetic circuit DAG wij : weight of edges from vi to vj

2. Example x1 x2 4 3 x1 = 4 x2 = 3 x3 = x1 + x2 x4 = x2 + x3 x5 = x2 * x4 x6 = x3 * x4 x7 = x4 * x6 x3 x4 + + x6 x5 * * Straight line code x7 * Arithmetic Circuit

3. degree v1 : x1 1 v2 : x2 1 v3 : x1 + x2 1 v4 : x1 + 2x2 1 v5 : x1x2 + 2x22 2 v6 : x12 + 3x2 + 2x22 v7 : x1 + … . . . Multi Variate Polynomial Parallel Algorithm O(logN log dN) d: degree of circuit (degree of polynomial) N: Size or number of nodes in the circuit d may be large (exponential of N) useful only if d is polynomial of N

4. Degree of poly O(N) Gaussian Elimination straight line program with O( N3) nodes Application O ( log2 N) time Construction of heap Sorting - Straight line program with O(N2) nodes O ( log2 N) time [bubble sort, insertion sort]

5. vj vj vi vj vi vj 1 1 vk * => vk Case (a) Both vi and vj are are leaves leaf node: its value is evaluated completely Eval - Add: all inputs are leves Eval - Mult: all inputs are leaves one input is leaf Skip - Add: not all inputs are leaves Eval - Add vi vj vi vj wjk wik => vk + vk Action of Eval -Add vk = viwik + vjwjk Eval - Mult vi vi wjk= value(vi) 1 1 * vk => + vk Case (b) vi is a leaf, and vj are not leaf (exactly one input is a leaf node)

6. Skip Add w2 w2 w3 w3 w1 w4 w1 w4 + + v w3*w5 w5 w2*w5 z z w1*w5 w4*w5 + + x Inputs of the first addition node x are promoted to become inputs to the second addition node y. Inputs to x are are maintained if x is an input to another node z.

7. Skip-Add Multiple edges from vi to vj may be inserted. New weight from vi to vj if vi, vj are addition node i original wij must be removed(vi, vj both + nodes) vj : addition node vi : not k vj is not an addition nde j if vk, vj are addition node otherwise Skip Add: W <- W - W+ + WW+ O(n3) PEs, O(logn) time

8. v1 6 2 v2 x1 = 6 x2 = 2 x3 = x1 + x2 x4 = x2 + x3 x5 = x3• x4 x6 = x3 + x5 x7 = x4• x5 v3 + + v4 v5 • • v7 + v6 Straight-line Code Arithmetic Circuit node polynomial degree x1 x2 x1 + x2 x1 + 2x2 x12+3x1x2+2x22 x12+3x1x2+2x22+x1+x2 x13+5x12x2+8x1x22+4x23 1 1 1 1 2 2 3 v1 v2 v3 v4 v5 v6 v7

9. v1 v2 v1 v2 v1 v2 6 2 6 2 6 2 1 1 8 1 v3 v3 v3 8 8 8 1 1 1 v4 v4 v4 + + + 1 8 1 8 1 1 v5 1 v5 v5 1 1 1 + + • 1 1 1 1 1 v7 v7 v7 • 8 • • v6 v6 + + + v6 after Eval-Add after Eval-Mult after Skip-Add v1 v2 v1 6 2 v2 v1 6 2 v2 6 2 v3 8 v3 8 v3 8 10 v4 10 v4 10 v4 1 v5 80 v5 v5 8 1 80 80 8 1 v7 v7 v7 • 8 v6 + v6 800 + v6 800 + after Eval-Add after Eval-Mult after Skip-Add

10. Complexity a b Define height leaf node = 2 * node : h(x) = h(a) + h(b) + node: max (a+1, m) a = max (h1,..., hi) v1,..., vi : + nodes inputs to x m = max (hi+1 ,..., hk) vi+1,....,vk : * nodes inputs to x degree and height: almost same except + node degree: max(d1,d2), height: max (h1,h2)+1 Theorem: O(logdN) phases are sufficient. Proof is based on the following two claims. Claim 1: The of an N-node circuit of degree d  (N+1)d Claim 2: After log3/2h phases, every node has been reduced to a leaf node. * x vi+1 v1 vi vk + * * + + x

11. Proof on the complexity of straight line program Lemma: The height of an arithmetic circuit of degree d is at most (Z+2)d, where Z is the number of addition nodes in the circuit that have addition nodes as inputs. Proof: By induction on the number of nodes N in the circuit. Basis (N=1) trivially true. Inductive hypothesis: Assume that the claim is true for N. Inductive step: Let us prove for N+1. Let v be a node with degree d and max height h with inputs v1,...,vk, having degree d1,...dk, and heights h1,...,hk. Three cases. case 1: v is “*” node. Then d=d1 + ... + dk, and h=h1 + ... + hk, hi (Z+2)di by inductive hypothesis. Thus, h  (Z+2)d. case 2: v is “+” node, and max of h1 ,..., hkis not attained by “+” node. hi = max{hi} max{(Z+2)di} (Z+2)d. case 3: v is “+” node, and max of h1 ,..., hkis attained by “+” node. hi = 1 + max{hi} 1 + max{(Z+1)di} 1 + (Z+1)d  (Z+2)d. ( hi (Z+1)di follows from the fact that v itself is a type of Z, and must not be part of di subcircuit, and inductive hypothesis.) Thus, the lemma holds. Corollary: The height of N node circuit of degree d is at most (N+1)d. Proof: Since Z  N-1, it follows.

12. v2 v1 * Proof on the complexity (cont’) Lemma: Given any circuit and node v, let h :the height of v at the beginning of a phase, h’ :the height of v at the end of the phase. If h  3, then h’  (2/3)h. Moreover, if v is *” node that gets converted into an addition node during the phase, then h’  (2/3)h - 1/3. Proof: Induction on the size of circuit. Suppose that v is *node at the beginning of phase. Case 1. v becomes a leaf during the phase. h’=2, * node h  4. Thus, h’  h/2. Case 2. v remains *. Nor v1, v2 are leaf before Eval-Mult. Thus, h1,h2 > 3 h1’  (2/3)h1, h2’  (2/3)h2, by inductive hypo. Thus, h’  h1’ + h2’  (2/3) (h1 +h2 )  (2/3)h v Case 3. v becomes “+” node during the phase. One node was leaf, the other is not. Let v1 be the input to v before the phase. After the phase, either v1 stays as the only input, or v1 participate skip-add during the phase. If v1 remains input of v: h = h1+2, h1’ (2/3)h1, thus, h’ = h1’+1  (2/3)h- 1. If v1participate skip-add during the phase, then any new input u to v after the phase should be the input to v1 before the phase. Note that the height of u is less than the height of v1, so the same argument as the above holds.