Decision Procedures An Algorithmic Point of View

1. Decision ProceduresAn Algorithmic Point of View Gaussian Elimination andSimplex Daniel Kroening and Ofer Strichman

2. Gaussian’s elimination • Given a linear system Ax = b • Manipulate A|b to an upper-triangular form A x = b

3. Gaussian’s elimination • Then, solve backwards from the k’s row according to:

4. Gaussian elimination - example And now… x3 = -1, x2 = 3, x1 = 1 problem solved.

5. Feasibility with Simplex • Simplex was originally designed for solving the optimization problem: s.t. • We are only interested in the feasibility problem. Is this system feasible ? Is this system optimal ?

6. General simplex • We will learn a variant called general simplex. • Very suitable for solving the feasibility problem fast. • The input: • A is a m£n coefficient matrix • The problem variables: • First step: convert the input to general form

7. General form • General form: • A combination of: • Linear equalities of the form • Lower and upper bounds on variables.

8. Converting to General Form • A: Replace (where )with and • s1,..., sm are called the additional variables.

9. Example 1 • Convertto: It is common to keep the conjunctions implicit

10. Example 2 • Convertto:

11. Simplex basics… • Linear inequality constraints, geometrically, define a convex polyhedron. © Wikipedia

12. Our example from before, geometrically General Simplex begins in the origin...

13. Matrix form • Recall the general form: • Due to the additional variables: • now A is an m£ (n + m) matrix. xy s1 s2 s3

14. xy s1 s2 s3 xy s1 s2 s3 The tableau • The diagonal part is inherent to the general form • We can instead write: This is called the tableau

15. xy s1 s2 s3 The tableau • The tableau changes throughout the algorithm, but maintains its m £ n structure • Distinguish between basic and nonbasic variables • Initially, basic variables = the additional variables. Nonbasic variables Basic variables

16. The tableau • Denote by • B– Basic variables • N – Nonbasic variables • The tableau is simply a rewrite of the system: • The basic variables are also called the dependent variables.

17. The general simplex algorithm • Simplex maintains: • The tableau, • an assignment ® to all variables • The bounds • Initially, • B = additional variables • N = problem variables • ®(xi) = 0 for i2 {1,...,n+m}

18. Invariants • Two invariants are maintained throughout: • All nonbasic variables satisfy their bounds • Can you see why these invariants are maintained initially ? • We should check that they are indeed maintained

19. The general simplex algorithm • The initial assignment satisfies • If the bounds of all basic variables are satisfied by ®, return `Satisfiable’. • Otherwise... pivot.

20. Pivoting • Find a basic variable xithat violates its bounds. • Suppose that ®(xi) < li • Find a nonbasic variable xj such that • aij > 0 and(xj) < uj, or • aij < 0 and(xj) > lj • Why ?

21. Pivoting • Find a basic variable xithat violates its bounds. • Suppose that ®(xi) < li • Find a nonbasic variable xj such that • aij > 0 and(xj) < uj, or • aij < 0 and(xj) > lj • Such a variable xj is called suitable. • If there is no suitable variable – return ‘Unsatisfiable’ • Why ?

22. Pivoting xiwithxj • Solve equation i for xj: From: To: • Swap xi and xj, and update the i-th row accordingly. From To:

23. Pivoting xi withxj • Update all other rows: • Replace xjwith its equivalent obtained from row i:

24. Pivoting • Update ®as follows: • Increase ®(xj) by • Now xj is a basic variable: it can violate its bounds. • Update ®(xi) accordingly • Q: What is now ®(xi) ? • Update  for all other basic (dependent) variables.

25. Example • Recall the tableau and constraints in our example: • Initially  assigns 0 to all variables • Bounds of s1 and s3 are violated

26. Example • Recall the tableau and constraints in our example: • We will solve s1 • x is a suitable nonbasic variable for pivoting • It has no upper bound • So now we pivot s1 with x

27. Example • Recall the tableau and constraints in our example: • Solve 1st row for x: • Replace x with s1 in other rows:

28. Example • The new state: • Solve 1st row for x: • Replace x with s1 in other rows:

29. Example • The new state: • What about the assignment ? • We should increase x by • Hence, (x) = 0 + 2 = 2 • Now s1is equal to its lower bound: (s1) = 2 • Update all the others

30. Example • The new state: • Now s3 violates its lower bound • Which nonbasic variable is suitable for pivoting ? • That’s right… y

31. Example • The new state: • We should increase y by

32. Example • The final state: • All constraints are now satisfied

33. Observations • The additional variables: • Only additional variables have bounds. • These bounds are permanent. • Additional variables exit the base only on extreme points (their lower or upper bounds). • When entering the base, they shift towards the other bound and possibly cross it (violate it).

34. Observations • Can it be that we pivot(xi,xj) and then pivot(xj,xi) and enter a (local) cycle ? • No. • For example, suppose that aij > 0. • We increased (xj) so now (xi) = li. • After pivoting, possibly (xj) > uj • But aij’ = 1 / aij > 0, hence xi is not suitable.

35. Observations • Is termination guaranteed ? • Not obvious. • Perhaps there are bigger cycles. • In order to avoid circles, we use Bland’s rule: • determine a total order on the variables. • Choose the first basic variable that violates its bounds, and first nonbasic suitable variable for pivoting. • It can be proven that this guarantees that no base is repeated, which implies termination.

36. General-Simplex with Bland’s rule