Dynamic Programming

Dynamic Programming Jeff Chastine

Dynamic Programming • Usually involves optimization problems • Optimal solution exists • Sub-problems arise more than once • Could use brute force, but… • Main idea: If you’ve already solved the sub-problem, leave yourself a note! Jeff Chastine

To Get You Thinking… • Fibonacci How would you calculate the 100th Fibonacci number? Fib (100) = Fib(99) + Fib(98) Fib (99) = Fib (98) + Fib (97) Jeff Chastine

Assembly Line Scheduling line 1 a1,1 a1,2 a1,3 a1,n-1 a1,n e1 x1 t1,1 t1,2 t1,n-1 … finishedcar chassis t2,1 t2,2 t2,n-1 e1 x2 a2,1 a2,2 a2,3 a2,n-1 a2,n line 2 Jeff Chastine

Assembly Line Scheduling line 1 a1,1 a1,2 a1,3 a1,n-1 a1,n e1 x1 t1,1 t1,2 t1,n-1 … finishedcar chassis t2,1 t2,2 t2,n-1 e1 x2 a2,1 a2,2 a2,3 a2,n-1 a2,n line 2 Entry Time – time to get the chassis on the line Jeff Chastine

Assembly Line Scheduling line 1 a1,1 a1,2 a1,3 a1,n-1 a1,n e1 x1 t1,1 t1,2 t1,n-1 … finishedcar chassis t2,1 t2,2 t2,n-1 e1 x2 a2,1 a2,2 a2,3 a2,n-1 a2,n line 2 Station Time – time in each station. Note a1,x != a2,x Jeff Chastine

Assembly Line Scheduling line 1 a1,1 a1,2 a1,3 a1,n-1 a1,n e1 x1 t1,1 t1,2 t1,n-1 … finishedcar chassis t2,1 t2,2 t2,n-1 e1 x2 a2,1 a2,2 a2,3 a2,n-1 a2,n line 2 Transfer Time – time to move to the other assembly line Jeff Chastine

Assembly Line Scheduling line 1 a1,1 a1,2 a1,3 a1,n-1 a1,n e1 x1 t1,1 t1,2 t1,n-1 … finishedcar chassis t2,1 t2,2 t2,n-1 e1 x2 a2,1 a2,2 a2,3 a2,n-1 a2,n line 2 Exit Time – time to get the chassis off the line Jeff Chastine

What’s the fastest way through? line 1 a1,1 a1,2 a1,3 a1,n-1 a1,n e1 x1 t1,1 t1,2 t1,n-1 … finishedcar chassis t2,1 t2,2 t2,n-1 e1 x2 a2,1 a2,2 a2,3 a2,n-1 a2,n line 2 We have a rush order! Jeff Chastine

What about Brute Force? • Try all paths • How many? 2n • This is the lower bound! • Example: 100 stations? • You’ll be dead before it finishes calculating • Good gift for your grandchildren Jeff Chastine

Giving the problem structure! • There’s only 1 path to the first station • There are 2 paths to the subsequent stations • For station S1, j: • S1, j-1 OR • S2, j-1 with transfer time t2, j-1 Jeff Chastine

What’s the fastest way through? S1,1 S1,2 S1,3 S1,4 S1,5 S1,6 line 1 4 8 4 7 3 9 3 2 1 3 2 3 3 4 finishedcar chassis 1 2 2 1 2 2 4 8 6 5 4 5 7 line 2 S2,1 S2,2 S2,3 S2,4 S2,5 S2,6 Jeff Chastine

What’s the fastest way through? S1,1 S1,2 S1,3 S1,4 S1,5 S1,6 line 1 4 8 4 7 3 9 3 2 1 3 2 3 3 4 finishedcar chassis 1 2 2 1 2 2 4 8 6 5 4 5 7 line 2 S2,1 S2,2 S2,3 S2,4 S2,5 S2,6 f1[j] f1[2] = min ((9+9), (12+2+9)) = 18 from line 1 f2[j] f2[2] = min ((12+5), (9+2+5)) = 16 from line 1 Jeff Chastine

What’s the fastest way through? S1,1 S1,2 S1,3 S1,4 S1,5 S1,6 line 1 4 8 4 7 3 9 3 2 1 3 2 3 3 4 finishedcar chassis 1 2 2 1 2 2 4 8 6 5 4 5 7 line 2 S2,1 S2,2 S2,3 S2,4 S2,5 S2,6 l1[j] f1[j] l2[j] f2[j] Jeff Chastine

What’s the fastest way through? S1,1 S1,2 S1,3 S1,4 S1,5 S1,6 line 1 4 8 4 7 3 9 3 2 1 3 2 3 3 4 finishedcar chassis 1 2 2 1 2 2 4 8 6 5 4 5 7 line 2 S2,1 S2,2 S2,3 S2,4 S2,5 S2,6 f1[j] f1[3] = min ((18+3), (16+1+3)) = 20 from line 2 f2[j] f2[3] = min ((16+6), (18+3+6)) = 22 from line 2 Jeff Chastine

What’s the fastest way through? S1,1 S1,2 S1,3 S1,4 S1,5 S1,6 line 1 4 8 4 7 3 9 3 2 1 3 2 3 3 4 finishedcar chassis 1 2 2 1 2 2 4 8 6 5 4 5 7 line 2 S2,1 S2,2 S2,3 S2,4 S2,5 S2,6 l1[j] f1[j] l2[j] f2[j] Jeff Chastine

What’s the fastest way through? S1,1 S1,2 S1,3 S1,4 S1,5 S1,6 line 1 4 8 4 7 3 9 3 2 1 3 2 3 3 4 finishedcar chassis 1 2 2 1 2 2 4 8 6 5 4 5 7 line 2 S2,1 S2,2 S2,3 S2,4 S2,5 S2,6 l1[j] f1[j] f*=1 f*=38 l2[j] f2[j] Jeff Chastine

Matrix Multiplication • [p×q][q×r] →[p×r] for pqr scalar operations • Most efficient way to multiply?<A1, A2, A3, A4>(A1, (A2, (A3, A4)))(A1, ((A2, A3), A4))((A1, A2), (A3, A4))((A1, (A2, A3)), A4)(((A1, A2), A3), A4) Example:[10×100][100×5][5×50](([10×100][100×5])[5×50]) // 5000 (([10×5])[5×50]) // 2500([10×100]([100×5][5×50])) // 25000 ([10×100]([100×50])) // 50000 Jeff Chastine

How many Parenthesizations?(say that 5 times fast) For any sequence of matrices, we can split at k M1M2M3M4 … MkMk+1Mk+2 …Mn Jeff Chastine

Optimal Substructure For any sequence of matrices, we can split at k M1M2M3M4 … MkMk+1Mk+2 …Mn M1M2…Mk must be optimal Mk+1Mk+2…Mnmust be optimal as well // Note the recursion! Jeff Chastine

Recursive Solution Let m[i, j] be the min # of multiplications m[1, n] is cheapest way to compute solution Then, m[i, j] = m[i, k] + m[k+1, j] + pi-1 pkpj Cost to multiply matrices together Big problem. We don’t know k! Jeff Chastine

Recursive Solution Let m[i, j] be the min # of multiplications m[1, n] is cheapest way to compute solution Then, m[i, j] = min {m[i, k] + m[k+1, j] + pi-1 pkpj} Jeff Chastine

6 1 Matrix M1 M2M3 M4M5M6 Dimension 30 × 35 35 × 15 15 × 5 5 × 10 10 × 20 20 × 25 5 2 4 3 3 4 ? 2 5 2,625(35x5) 750(15x10) 1,000(5x20) 5,000(10x25) 15,750(30x15) 6 1 0 0 0 0 0 0 M1 M4 M2 M3 M5 M6 Jeff Chastine

6 1 Matrix M1 M2M3 M4M5M6 Dimension 30 × 35 35 × 15 15 × 5 5 × 10 10 × 20 20 × 25 5 2 4 3 3 4 7875(30x5) 2 5 2,625(35x5) 750(15x10) 1,000(5x20) 5,000(10x25) 15,750(30x15) 6 1 0 0 0 0 0 0 M1 M4 M2 M3 M5 M6 Jeff Chastine

6 1 Matrix M1 M2M3 M4M5M6 Dimension 30 × 35 35 × 15 15 × 5 5 × 10 10 × 20 20 × 25 5 2 4 3 ? 3 4 2500(15x20) 3500(5x25) 4375(35x10) 7875(30x5) 2 5 2,625(35x5) 750(15x10) 1,000(5x20) 5,000(10x25) 15,750(30x15) 6 1 0 0 0 0 0 0 M1 M4 M2 M3 M5 M6 Jeff Chastine

6 1 Matrix M1 M2M3 M4M5M6 Dimension 30 × 35 35 × 15 15 × 5 5 × 10 10 × 20 20 × 25 5 2 4 3 7125 3 4 2500(15x20) 3500(5x25) 4375(35x10) 7875(30x5) 2 5 2,625(35x5) 750(15x10) 1,000(5x20) 5,000(10x25) 15,750(30x15) 6 1 0 0 0 0 0 0 M1 M4 M2 M3 M5 M6 Jeff Chastine

6 1 Matrix M1 M2M3 M4M5M6 Dimension 30 × 35 35 × 15 15 × 5 5 × 10 10 × 20 20 × 25 ? 5 2 11875(30x20) 10500(35x25) 4 3 9375(30x10) 7125(35x20) 5375(15x25) 3 4 2500(15x20) 3500(5x25) 4375(35x10) 7875(30x5) 2 5 2,625(35x5) 750(15x10) 1,000(5x20) 5,000(10x25) 15,750(30x15) 6 1 0 0 0 0 0 0 M1 M4 M2 M3 M5 M6 Jeff Chastine

6 1 Matrix M1 M2M3 M4M5M6 Dimension 30 × 35 35 × 15 15 × 5 5 × 10 10 × 20 20 × 25 15125(30x25) 5 2 11875(30x20) 10500(35x25) 4 3 9375(30x10) 7125(35x20) 5375(15x25) 3 4 2500(15x20) 3500(5x25) 4375(35x10) 7875(30x5) 2 5 2,625(35x5) 750(15x10) 1,000(5x20) 5,000(10x25) 15,750(30x15) 6 1 0 0 0 0 0 0 M1 M4 M2 M3 M5 M6 Jeff Chastine

Summary • Elements of Dynamic Programming • Bottom-up approach • Optimal substructure • Fastest way through station j contained fastest way through station (j-1) • Determining correct split contained optimal solutions to subproblems • Overlapping subproblems • Doesn’t solve the same subproblems over and over Jeff Chastine

A Note about Memoization • Top-down approach • Leave a note about solutions to sub-problems • Example: Fibonacci numbers 1, 1, 2, 3, 5, 8, 13… Calculate the nth Fibonacci number Jeff Chastine

Why this could be bad F(10) + F(8) F(9) Jeff Chastine

Why this could be bad F(10) F(8) F(9) F(6) F(7) F(7) F(8) Jeff Chastine

Why this could be bad F(10) height = ~10 F(8) F(9) F(6) F(7) F(7) F(8) F(4) F(5) F(5) F(6) F(5) F(6) F(6) F(7) Ballpark: 210 nodes! Jeff Chastine

What to notice F(10) F(8) F(9) F(6) F(7) F(7) F(8) F(4) F(5) F(5) F(6) F(5) F(6) F(6) F(7) Look at the redundancy! Jeff Chastine

Memoization • Leave a note • Once you solve a problem • Write it in a table • Instead of calculating/branching again, look it up! Jeff Chastine

Dynamic Programming