Optimization Techniques Constrained Optimization by Linear Programming updated 10.11.01

1. SMU EMIS 5300/7300 NTU SY-521-N Systems Analysis Methods Dr. Jerrell T. Stracener, SAE Fellow Optimization Techniques Constrained Optimizationby Linear Programming updated 10.11.01

2. Nature of Linear Programming

3. The Product Mix Problem • Among the most common in linear programming • The problem is to find out which products to • include in the production plan and in what quantities • these should be included (product mix) in order • to maximize the profit. • A solution tells more generally, in effect, how to • allocate scarce resources.

4. The Blending Problem • Involve the determination of the best blend of • available ingredients to form a certain quantity of a • product under strict specifications required to meet • a designated level of output or given specifications • Especially important in the process industries such • as petroleum, chemicals and food, and in fields where • a certain level of service is desired at minimum cost • In blending, the cost of the ingredients is to be • minimized , while adhering to certain specifications • and using certain ingredients (resources).

5. The Blending Problem • An attempt is made to use as few resources as • possible to provide a given product or service level • Note: A blending problem is also considered a • problem of allocating resources in the best manner • and in both cases we try to achieve the highest • ratio of outputs to inputs; that is, to maximize • productivity.

6. Formulation of the Linear Programming Model • Management science models, are composed of • three components: • The decision (controllable) variables • The environment (uncontrollable) variables • The result (dependent) variables • The LP model is composed of the same • components, but they assume different names

7. Decision Variables The decision variables in LP depend on the type of LP problem being considered. They can be the quantities of the resources to be allocated, or the number of units to be produced. The decision maker is searching for the value of these unknown variables which will provide an optimal solution to the problem. The Decision Variables (controllable) Objective Function (result variables) The Constraints (uncontrollable variables and relationships)

8. The Objective Function • A linear programming model attempts to optimize • a single goal, written as a linear function; for example • maximize 5x1 + 7x2 + x3. That is, it attempts to • find either the maximum level of a desirable goal, • such as total share of the market or total profit, or • the minimum level of some undesirable outcome, • such as total cost. • Two Step Approach • 1. Select a primary goal whose level is to be • maximized or minimized. • 2. Transform the other goals into constraints, • which must only be satisfied.

9. The Constraints • The decision maker is searching for the values • of the values of the decision variables that will • maximize (or minimize) the value of the objective • function. Such a process is usually subject to • several uncontrollable restrictions, requirements, • or regulations that are called constraints. These • constraints are expressed as linear equations and/or • inequalities.

10. Formulation of a Linear Programming Model • Every LP problem is composed of: • Decision variables - The variables whose values are • unknown and are searched for. Usually they are • designated by x1, x2, … • Objective function - This is a mathematical • expression, given as a linear function, that shows • the relationship between the decision variables and • a single goal (or objective) under consideration. The • objective function is a measure of goal attainment.

11. Formulation of a Linear Programming Model • Optimization - Linear programming attempts to • either maximize or minimize the values of the • objective function. • Profit or cost coefficients - The coefficients of the • variables in the objective function are called the • profit (or cost) coefficients. They express the rate • at which the value of the objective function increases • by including in the solution one unit of each of the • decision variables.

12. Formulation of a Linear Programming Model • Constraints - The maximization (or minimization) • is performed subject to a set of constraints. • Therefore, linear programming can be defined as a • constrained optimization problem. These constraints • are expressed in the form of linear inequalities (or, • sometimes inequalities). They reflect the fact that • resources are limited (e.g., in a product-mix • problem) or the specific product requirements • (e.g., in a blending problem).

13. Formulation of a Linear Programming Model • Input-output (technology) coefficients - The • coefficients of the constraints’ variables are called • the input-output coefficients. They indicate the rate • at which a given resource is depleted or utilized. They • appear on the left hand side of the constraints. • Capacities - The capacities (or availability) of the • various resources, usually expressed as some upper • or lower limit, are given on the right hand side of the • constraints. The right-hand side also expresses • minimum requirements. • Non-Negativity - It is required that only non- • negative (zero or positive) values of the decision • variables be considered.

14. Formulation of a Linear Programming Model • The general linear programming model can be • presented in the following mathematical terms • aij = input-output coefficients • cj = cost (profit) coefficients • bi = capacities (right hand side) • xj = decision variables • Find a vector (x1, …, xn) which minimizes (or • maximizes) a linear objective function F(x) where: • F(x) = c1x1 + c2x2 + … + cjxj + … + cnxn

15. Formulation of a Linear Programming Model subject to the linear constraints: a11x1 + a12x2 + … + a1nxn b1 a21x1 + a22x2 + … + a2nxn b1 … … … … ai1x1 + ai2x2 + … + ainxn bi … … … … am1x1 + am2x2 + … + amnxn bm and the non-negativity constraints: x1 0, x2  0, …

16. Advantages of Linear Programming Model • Linear programming is a tool that can be used • to solve allocation type problems. Such problems • are very common and extremely important. • Solution is difficult due to the fact that an infinite • number of possible solutions may exist. • Linear programming not only provides the optimal • solution but does so in a very efficient manner. • It provides additional information concerning the • values of the resources that are allocated.

17. Limitations of Linear Programming Due to Assumptions • The applicability of linear programming is limited by • several assumptions. As in all mathematical models, • assumptions are made for reducing the complex • real-world problem into a simplified form. The major • ones are: • Certainty - It is assumed that all data involved in • the linear programming problem are known with • certainty • Linear objective function - It is assumed that the • objective function is linear. This means that per unit • cost, price or profit are assumed to be unaffected by • changes in production methods or quantities • produced or sold.

18. Limitations of Linear Programming Due to • Assumptions • Linear constraints - The constraints are also • assumed to be linear. This means that all the • input-output coefficients are considered to be • unaffected by a change of methods, quantities, • or utilization level. • Nonnegativity - Negative activity levels (or • negative production) are not permissable. It is • required, therefore, that all decision variables • take nonnegative values.

19. Limitations of Linear Programming Due to Assumptions • Additivity - It is assumed that the total utilization • of each resource is determined by adding together • that portion of the resource required for the • production of each of the various products or • activities. The assumption of additivity also means • that the effectiveness of the joint performance of • activities, under any circumstances, equals the • sum of the effectiveness resulting from the individual • performance of these activities.

20. Limitations of Linear Programming Due to Assumptions • Divisibility - Variables can, in general, be classified • as continuous or discrete. Continuous variables are • subject to measurement (e.g., weight), temperature • Independence - Complete independence of • coefficients is assumed, both among activities • and among resources. For example, the price of • one produce has no effect on the price of another.

21. Limitations of Linear Programming Due to Assumptions • Proportionality - The requirement that the • objective function and constraints must be linear is • a proportionality requirement. This means that the • amount of resources used, and the resulting value • of the objective function, will be proportional to the • value of the decision variables.

22. Solving Linear Programs • All solutions to a linear programming problem • which satisfy all the constraints are called feasible • The collection of feasible solutions is called the • feasible solution space or area • Any solution which violates one or more of the • constraints is termed infeasible

23. Solving Linear Programs • The major task in applying linear programming is • the formulation of the problem. • Once a problem has been formulated, one of • several available methods of solution can be • applied • The graphical method, whose main purpose is to • illustrate the concepts involved in the solution process • The general, computationally powerful simplex • method • Other approaches

24. The Graphical Method of Solution The graphical method is used mainly to illustrate certain characteristics of LP problems and to help in explaining the simplex method. The only case where it has a practical value is in the solution of small problems with two decision variables and only a few constraints, or, problems with two constraints or only a few decision variables.

25. Example The two models of color sets produced by the Sekido Corporation, will be designated as A and B. The company is in the market to make money; that is, its objective is profit maximization. The profit realized is $300 from set A and $250 from set B. Obviously, the more A sets produced and sold, the better. The trouble is that there are certain limitations which prevent Sekido Company from producing and selling thousands of sets. These Limitations are: 1. Availability of only 40 hours of labor each day in the production department (a labor constraint). 2. A daily availability of only 45 hours of machine time (a machining constraint). 3. Inability to sell more than 12 sets of model A each day (a marketing constraint). (the constraints are on the next slide)

26. Example First let A = x1 and B = x2. So we need to maximize z = 300x1 + 250x2 with these constraints 2x1 + 1x2 40 (labor constraint) 1x1 + 3x2 45 (machining constraint) 1x1 + 0x2 12 (marketing constraint) x1 0 nonnegativity x2 0 nonnegativity

27. Example - solution • Graphing the feasible area - The graphical method • starts with the graphing of a feasible are within which • a search for the optimal solution is to be conducted. • The feasible area is established through graphing all • of the inequalities and equations that describe the • constraints. 2nd quadrant infeasible x1  0, x2  0 1st quadrant feasible x1  0, x2  0 3rd quadrant infeasible x1  0, x2  0 4th quadrant infeasible x1 0, x2 0

28. Example - solution • Graphing the first constraint - This is expressed • as 2x1 + x2 40. The steps in drawing the constraint are: • Step 1: Since an inequality of the type less than • or equal to has in effect two parts, we will first • consider only the equality part of the constraint. • In our example, it will be 2x1 + x2= 40. Since an • equation can be shown graphically as a straight line, it is sufficient to find the coordinates of two points to graph the entire equation as a line. To do so, x1 is first set to zero. This will yield a point where the equation intersects the x2 axis.

29. Example - solution if we repeat this for the rest of the constraints we will get a graph that looks like this:

30. a 15 b d c 0 12 Example - solution We limit our selves to the largest region that satisfies all our constraints. This region has 4 vertices, which we will use in our main equation to try and maximize it. maximum

31. Example - solution So our maximum value of $6,350 is when x1 = 12 and x2 =11.

32. SMU EMIS 5300/7300 NTU SY-521-N Systems Analysis Methods Dr. Jerrell T. Stracener, SAE Fellow LP Solutions The Graphical Method prepared and presented by Imran Ismail updated 10.11.01

33. Consider the following LP Maximize X + 2Y Subject to: X + Y <= 4 X - 2Y <= 2 -2X + Y <= 2 X, Y >= 0 This problem is in 2-dimensions and can be solved graphically

34. Finding the Feasible Region We begin by graphing the constraints on an X-Y coordinate system to determine the set of all points that satisfy all the constraints. This set of points is known as the feasible region for the LP. Since both variables must be non-negative, we know that the feasible region must be within the first quadrant.

35. The constraint lines can be constructed by changing all inequalities in to equal signs. i.e. X + Y <= 4 become X + Y = 4, the inequalities play an important role in defining the feasible region and are implemented later on.

36. The set of solutions to the equation X + Y = 4 can be represented by the straight line passing through the points (4,0) and (0,4) in the XY plane

37. Similarly for constraint X-2Y <= 2, we get

38. And for constraint 3, -2X+Y <= 2: Since X and Y must be nonnegative, the feasible region for the LP is the bounded by the three lines described (and shown) above and the X and Y axes. Any point inside or on the boundary of the region described above is a feasible solution to the LP

39. Finding an Optimal Solution The objective function is given as an equation whose value has yet to be determined. The information provided in the LP gives us 2 clues: • The function has to be maximized • The function must satisfy all constraints i.e. it must lie in the feasible regions

40. In such an instance, we can start by assuming an objective function value of 0 (zero). This gives us the following equation X + 2Y = 0 The line above will provide us with a set of points that will have an objective function value of 0. When drawn on the graph, the segment of this line that intersects the feasible region will represent the set of feasible solutions with an objective function value of zero. In this case, the set is just the single point (0,0).

41. Similarly, the line X + 2Y = 2 describes the set of points that have an objective function value of two. So, the segment of this line that intersects the feasible region represents the set of feasible solutions with an objective function value of two. We can continue this method till we reach an objective function value such that it no longer intersects the feasible region.

42. By continuing on in this fashion, we can find an optimal solution for the LP by "pushing" the objective function line up until it last touches the feasible region.

43. This occurs when we graph the line X + 2Y = 22/3 = 7.333 which intersects the feasible region at the point (2/3, 10/3). Since there are no feasible solutions with a greater objective function value than 22/3, we say that X = 2/3, Y = 10/3 is an optimal solution and that 22/3 is the optimal value for the objective function.

44. It is important to investigate how other objective functions might behave given the same feasible region.

45. It is observed that objective functions with different slopes, while traveling through the feasible region might be maximized at different points. It is also important to note that the corner points of the feasible region are the only part of this region that do not allow the objective function to move further ahead while still remaining feasible! Therefore, if an LP has an optimal solution, it is guaranteed that this solution will occur at a corner or extreme point of the feasible region

46. The “Brute Force” Method Using the fact that if an LP has an optimal solution, then it has an extreme point solution, we can use a "brute force" method to find an optimal solution by: • testing each extreme point to see if it is feasible and then comparing the objective function values. Recall that an extreme, or corner-point, solution is formed by the intersection of two of the constraints.

47. In our example we can identify the extreme points by labeling A, B, C, D and E, such that: A E B D C

48. Point A is formed by the intersection of: X + Y = 4 and –2X + Y = 2 and similarly: B: X - 2Y = 2 and X + Y = 4 C: X - 2Y = 2 and Y = 0 D: X = 0 and Y = 0 E: -2X + Y = 2 and X = 0 Solving these equations and plugging them into the objective function, we get:

49. Here, the co-ordinate points of A give us the maximum objective function value

50. Point A is also where the objective function line was graphically maximized. Therefore, we conclude that point A is the optimal extreme point of the feasible region. It: • maximizes the value of the objective function and • satisfies all constraints The (x,y) co-ordinates of point A are called the optimal decision variables or the optimal solution with optimal objective function value ZA