A Hard Problem: The TSP

1. Amirkabir University of Technology A Hard Problem: The TSP By: Mahmood Reza Jahanseir

2. Basic definitions • Lower bounds: Relaxations • The assignment relaxation • The MST relaxation • The s-tree relaxation • The LP relaxation • Lower bounds: Subgradient optimization • Approximation algorithms • Upper bounds: Heuristics • Upper bounds: Local search • Optimal solutions: Branch and bound Table of Contents

3. Basic definitions Upper bounds: Heuristics Upper bounds: Local search Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

4. Let w: E → R+ be a weight function on the complete graph Kn. We seek a cyclic permutation (1, π(1), . . . , πn−1(1)) of the vertex set {1, . . . , n} such that is minimal. We call any cyclic permutation π of {1, . . . , n} as well as the corresponding Hamiltonian cycle in Kn a tour; if w(π) is minimal among all tours, π is called an optimal tour. TSP is NP-complete, so that we cannot expect to find an efficient algorithm for solving it.

5. Metric Travelling Salesman Problem (ΔTSP): Let W = (wij) be a symmetric matrix describing a TSP, and assume that W satisfies the triangle inequality: wik≤ wij+ wjkfor i, j, k = 1, . . . , n. Then one calls the given TSP metric or, for short, a ΔTSP. • Theorem 1.ΔTSP is NP-complete. • Proof. Let G = (V,E) be a given connected graph, where V = {1, . . . , n}, and let Kn be the complete graph on V with weights: Obviously, G has a Hamiltonian cycle if and only if there exists a tour π of weight w(π) ≤ n (and then, of course, w(π) = n) in Kn. Thus the given polynomial algorithm for TSP allows us to decide HC in polynomial time.

6. In the metric case, there always exists a reasonably good approximation algorithm; most likely, this does not hold for the general case, where the triangle inequality is not assumed. • Asymmetric Travelling Salesman Problem (ATSP). Instead of Kn, we consider the complete directed graph on n vertices: we allow the weight matrix W to be non-symmetric (but still with entries 0 on the main diagonal). This asymmetric TSP contains the usual TSP as a special case, and hence it is likewise NP-hard.

7. Example 1. We drop the condition that the travelling salesman should visit each city exactly once, so that we now consider not only Hamiltonian cycles, but also closed walks containing each vertex of Kn at least once. If the given TSP is metric, any optimal tour will still be an optimal solution. However, this does not hold in general, as the example given in following figure shows: here (w, x, y, z, x,w) is a shortest closed walk (of length 6), but the shortest tour (w, x, y, z,w) has length 8.

8. Given a matrix W = (wij) not satisfying the triangle inequality, we may consider it as a matrix of lengths on Kn and then calculate the corresponding distance matrix D = (dij ). For example, we can use the algorithm of Floyd and Warshall for this purpose. Of course, D satisfies the triangle inequality and, hence, defines a metric TSP. It is easy to see that the optimal closed walks with respect to W correspond to the optimal tours with respect to D. Thus the seemingly more general problem described in Example 1 actually reduces to the metric TSP. One may also consider an arbitrary connected graph G with some length function w instead of Kn. Then it is not at all clear whether any tours exist: we need to check first whether G is Hamiltonian. this feasibility question is already an NP-complete problem in itself.

9. Lower bounds: Relaxations Upper bounds: Heuristics Upper bounds: Local search Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

10. In order to judge the quality of an approximate solution, we need lower bounds on the length of a tour, and these bounds should not only be strong but also easily computable. • A standard approach is the use of suitable relaxations: instead of the original problem P, we consider a problem P’ containing P; this auxiliary (simpler) problem is obtained by a suitable weakening of the conditions defining P. • Then the weight w(P’) of an optimal solution for P’ is a lower bound for the weight w(P) of an optimal solution for P. • Unfortunately, in many cases it is not possible to predict the quality of the approximation theoretically, so that we have to use empirical methods: for instance, comparing lower bounds found by relaxation with upper bounds given by solutions constructed by some heuristic.

11. Assignment problem. Let A be a given square matrix with nonnegative real entries. We require a diagonal of A for which the sum of all its entries is maximal (or minimal). Thus we seek a permutation π of {1, . . . , n} for which w1,π(1) + . . . + wn,π(n) becomes minimal. In particular, we have to examine all cyclic permutations π (each of which determines a tour); for these permutations, the sum in question equals the length of the associated tour. Therefore we can indeed relax TSP to AP. As we are not interested in permutations with fixed points for the TSP anyway, we can avoid this problem by simply putting wii = ∞ for all i. The assignment relaxation

12. Hungarian algorithm will allow us to do so with complexity O(n3). w(AP) is usually a reasonably good approximation to w(TSP) in practice – even though nobody has been able to prove this. Example 2. we replace the diagonal entries 0 in W by 88 to obtain the matrix W’. In order to reduce this AP to the determination of a maximal weighted matching, we consider the matrix W” = 88 − w’ijinstead of W’. Hungarian algorithm yields value 603. Any optimal matching for W is a solution of the original AP; hence w(AP) = 9×88−603 = 189. This gives the bound w(TSP) ≥ 189. W= W”=

13. Of course, a tour is not a tree; but if we omit any edge from a tour, we indeed get a spanning tree. This shows w(MST) ≤ w(TSP). An optimal solution for MST can be determined with complexity O(n2). Example 3. Determining a minimal spanning tree is also much easier than solving an AP. The MST relaxation w(T)=186

14. Let us choose a special vertex s ∈ V. An s-tree is a spanning tree for the induced subgraphKn \ s together with two edges incident with s. Obviously, every tour is a special s-tree; hence w(MsT) ≤ w(TSP), where MsT denotes the problem of determining a minimal s-tree. Note that it is easy to solve this problem: just determine a minimal spanning tree for Kn \ s, and add those two edges incident with s which have smallest weight. the resulting bound will usually depend on the choice of the special vertex s. The s-tree relaxation

15. Example 4. We choose s = Be. Hence w(TSP) ≥ w(B) = 186 + 44 = 230. Let T be a minimal spanning tree associated with the given TSP, and assume that s is a leaf of T. Then we obtain a minimal s-tree by adding an edge of smallest weight among all edges not in T and incident with s to T. In general, however, matters cannot be as simple.

16. there are (n − 1)n−3 distinct spanning trees on the remaining n−1 vertices. To each of these trees, we have to add one of the (n − 1)(n − 2)/2 pairs of edges incident with s, so that the total number of s-trees of Knis: 1/2 (n − 2)(n − 1)n−2 we will see that s-trees yield much better results when one also uses so-called penalty functions.

17. The assignment relaxation of the TSP can be described by the following ZOLP: Then the admissible matrices (xij) correspond precisely to the permutations in Sn. In order to restrict the feasible solutions to tours, we add the following subtour elimination constraints: The LP relaxation (15-1) (15-2)

18. Now let P be the polytope defined by the feasible solutions of previous inequalities; that is, the vertices of P correspond to the tours among the assignments. In principle, it is possible to describe P by a system of linear inequalities and solve the corresponding LP; unfortunately, the given inequalities do not suffice for this purpose. Even worse, nobody knows a complete set of corresponding inequalities, although large classes of required inequalities are known. Note that there is an exponential number of inequalities even in (15.2) alone.

19. Lower bounds: Subgradient optimization Upper bounds: Heuristics Upper bounds: Local search Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

20. We show how the lower bounds obtained from the s-tree relaxation can be improved considerably by using so-called penalty functions. This method was introduced by Held and Karp and used successfully for solving comparatively large instances of the TSP. The basic idea is rather simple: we choose some vector p = (p1, . . . , pn)T ∈ Rn and replace the weights wij of the given TSP by the transformed weights Let us denote the weight of a tour π with respect to the wij by w(π). hence any tour which is optimal for W is optimal also for W’.

21. On the other hand, the weight of an s-tree B is not transformed by just adding a constant: Thus the difference between the weight of a tour and the weight of an s-tree is: Where Let us assume that dp(B) is positive for every s-tree B. Then we can improve the lower bound w(MsT) of the s-tree relaxation with respect to W by determining a minimal s-tree with respect to W’: the gap between w(TSP) and w(MsT) becomes smaller according to (15.6).

22. Of course, it is not clear whether such a vector p exists at all, and how it might be found. We will use the following simple strategy: calculate a minimal s-tree B0 with respect to W, choose some positive constant c, and put Thus the non-zero coordinates of p impose a penalty on those vertices which do not have the correct degree 2 in B0. There remains the problem of choosing the value of c. It is possible to just use c = 1; however, in our example, we will select the most advantageous value (found by trial and error).

23. Example 5. we obtain p = (−3,−3, 0, 3, 0, 0,−3, 3, 3)T , where we have chosen c = 3. W’=

24. A minimal s-tree B1 with respect to W is displayed in following figure; its weight is w(B1) = 242. This time we choose c = 4 and p = (0,−4, 0, 0, 0, 0, 0, 0, 4)T. which yields the weight matrix W”. W”=

25. w(TSP) ≥ w(B2) = 248. This time we choose c = 1 and p=(0,−1, 0, 0, 0, 0, 1, 0, 0). This leads to the minimal s-tree B3 of weight w∗(B3) = 250

26. As B3 is actually a tour, we have now (coincidentally) solved the TSP. the following tour is optimal, and hence w(TSP) = 250. it would be nice to be able to choose the vector p as advantageously as possible. we want to minimize the gap d(p) between the length w’(TSP) of an optimal tour and the weight w‘(B) of a minimal s-tree B. If we want to minimize d(p), we need to determine

27. In general, we will not end up with L(w) = w(TSP): it is quite possible that no choice of p yields a minimal s-tree which is already a tour. In the metric case, the weight of an optimal tour is bounded by 3/2 times the Held-Karp lower bound. The vectors p are called subgradients in the general context. Unfortunately, one cannot predict how many steps will be required, so that the process is often terminated in practice as soon as the improvement between successive values becomes rather small. Held and Karp showed that (L) can also be formulated in terms of linear programming, and this yields, in practical applications, good bounds with moderate effort.

28. Approximation algorithms Upper bounds: Heuristics Upper bounds: Local search Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

29. Let P be an optimization problem, and let A be an algorithm which calculates a feasible – though not necessarily optimal – solution for any given instance I of P. We denote the weights of an optimal solution and of the solution constructed by A by w(I) and wA(I), respectively. If the inequality holds for each instance I, we call A an ε-approximative algorithm for P. Any connected Eulerianmultigraph on V is called a spanning Eulerianmultigraph

30. Lemma 1.Let W be the weight matrix of a ΔTSP on Kn, and let G = (V,E) be a spanning Eulerianmultigraph for Kn. Then one can construct with complexity O(|E|) a tour π satisfying w(π) ≤ w(E). Proof.We can determine with complexity O(|E|) an Euler tour C for G. Write the sequence of vertices corresponding to C in the form (i1, P1, i2, P2, . . . , in, Pn, i1), where (i1, . . . , in) is a permutation of {1, . . . , n} and where the P1, . . . , Pn are (possibly empty) sequences on {1, . . . , n}. Then (i1, . . . , in, i1) is a tour π satisfying

31. The easiest method is simply to double the edges of a minimal spanning tree. • Determine a minimal spanning tree T for Kn (with respect to the weights given by W). • Let G = (V,E) be the multigraph which results from replacing each edge of T with two parallel edges. • Determine an Euler tour C for G. • Choose a tour contained in C. Algorithm 1: tree algorithm

32. Example 6. We saw in Example 3 that the MST relaxation yields the minimal spanning tree T of weight w(T) = 186. A possible Euler tour for the doubled tree is: (Aa,Du,Ha,Be,Ha,Du,Fr,St,Ba,St,Nu,Mu,Nu,St, Fr,Du,Aa), which contains the tour of length 307:

33. Theorem 2.Algorithm 1 is a 1-approximative algorithm of complexity O(n2) for ΔTSP. Proof. Using the algorithm of Prim, step (1) has complexity O(n2). The procedure EULER can be used to perform step (3) in O(|E|) = O(n) steps. Clearly, steps (2) and (4) also have complexity O(n). This establishes the desired complexity bound. By Lemma 1, the tree algorithm constructs a tour π with weight w(π) ≤ 2w(T). On the other hand, the MST relaxation shows that all tours have weight at least w(T). Hence w(π) is indeed at most twice the weight of an optimal tour.

34. Next we present a 1/2 -approximative algorithm, which is due to Christofides. • Determine a minimal spanning tree T of Kn (with respect to W). • Let X be the set of all vertices which have odd degree in T. • Let H be the complete graph on X (with respect to the weights given by the relevant entries of W). • Determine a perfect matching M of minimal weight in H. • Let G = (V,E) be the multigraph which results from adding the edges of M to T. • Determine an Euler tour C of G. • Choose a tour contained in C. Algorithm 2: Christofides’ algorithm

35. Theorem 3.Algorithm 2 is a 1/2 -approximative algorithm of complexity O(n3) for ΔTSP. Proof: As G is Eulerian, the tour π determined in step (5) satisfies the inequality Thus we have to find a bound for w(M). Write |X| = 2m and let (i1, i2, . . . , i2m) be the vertices of X in the order in which they occur in some optimal tour σ. We consider the following two matchings of H: The triangle inequality for W implies since M is a perfect matching of minimal weight. Hence yields w(π) ≤ 3w(TSP)/2.

36. Example 7. let T be the minimal spanning tree with weight w(T) = 186. By inspection, we obtain M = {AaDu,BeMu,BaSt} with w(M) = 95. Euler tour which contains the tour Note that this tour has weight 266, which is only 6% more than the optimal value of 250.

37. Upper bounds: Heuristics • Upper bounds: Heuristics • Upper bounds: Local search • Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

38. Perhaps the most frequently used heuristics are the so-called insertion algorithms. • The current partial tour of length k is always extended to a tour of length k +1 by adding one more city in the k-th iteration; this involves two tasks: • choosing the city to be added and • deciding where the city chosen in (a) will be inserted into the current partial tour. • There are several standard strategies for choosing the city in (a): arbitrary choice; selecting the city which has maximal (or, alternatively, minimal) distance to the cities previously chosen; or choosing the city which is cheapest to add. • We also have to settle on a criterion for step (b); here an obvious strategy is to insert the city at that point of the partial tour where the least additional cost occurs.

39. Procedure FARIN(W, s;C) • C ← (s, s), K ← {ss}, w ← 0; • for u = 1 to n do d(u) ← wsu od • for k = 1 to n − 1 do • choose y with d(y) = max {d(u): u = 1, . . . , n}; • for e = ij ∈ K do c(e) ← wi,y + wy,j− wi,jod • choose an edge f ∈ K with c(f) = min {c(e): e ∈ K}, say f = uv; • insert y between u and v in C; • K ← (K \ {f}) ∪ {uy, yv}, w ← w + c(f), d(y) ← 0; • for x ∈ {1, . . . , n} \ C do d(x) ← min {d(x), wyx} • od Algorithm 3: Farthest Insertion

40. Example 8. We choose the vertex s = Fr as our starting point.

41. Upper bounds: Local search • Upper bounds: Heuristics • Upper bounds: Local search • Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

42. Having chosen a tour (at random or using a heuristic), the next step is to try to improve this tour as far as possible: we want to apply post-optimization. Suppose F is the set of all feasible solutions for a given optimization problem; for example, for the TSP, F would be the set of all tours. A neighborhood is a mapping N : F → 2F: we say that N maps each f ∈ F to its neighborhood N(f). Any algorithm which proceeds by determining local optima in neighborhoods is called a local search algorithm. Let f be a tour, and choose k ∈ {2, . . . , n}. The neighborhood Nk(f) is the set of all those tours g which can be obtained from f by first removing k arbitrary edges and then adding a suitable collection of k edges. One calls Nk(f) the k-change neighborhood. Any tour f which has minimal weight among all tours in Nk(f) is said to be k-optimal.

43. Choose an initial tour f. • while there exists g ∈ Nk(f) with w(g) < w(f) do • choose g ∈ Nk(f) with w(g) < w(f); • f ← g • od • Perhaps the most important problem concerns which value of k one should choose. • For large k (that is, larger neighborhoods), the algorithm yields a better approximation, but the complexity will grow correspondingly. In practice, the value k = 3 seems to work well. Algorithm 4: k-opt

44. Let f be a tour described by its edge set f = {e1, . . . , en}: Then the tours g ∈ N2(f) can be found as follows: ei and ej should not have a vertex in common. Note that every neighborhood N2(f) contains precisely n(n − 3)/2 tours g ≠ f.

45. δ(g) measures the advantage which the tour g offers compared to f. We set δ = max {δ(g): g ∈ N2(f)}; if δ > 0, we replace f by some tour g with δ(g) = δ. Otherwise, f is already 2-optimal, and the algorithm 2-opt terminates. each iteration has complexity O(n2); the number of iterations cannot be predicted.

46. Procedure 2-OPT(W, f; f) • repeat • δ ← 0, g ← f; • for h ∈ N2(f) do • if δ(h) > δ then g ← h; δ ← δ(h) fi • od • f ← g; • until δ = 0 Algorithm 5: 2-opt

47. Example 9. Let us choose the tour of weight 266 constructed using Christofides’ algorithm in Example 7 as our initial tour f. During the first iteration of 2-OPT, the edges BeMu and NuSt are replaced with BeNu and MuSt; this yields the following tour of length 253. The second iteration of 2-OPT exchanges the edges FrDuandAaHaforFrAaandDuHa. The resulting tour is the optimal tour of length 250

48. Optimal solutions: Branch and bound • Upper bounds: Heuristics • Upper bounds: Local search • Optimal solutions: Branch and bound • Basic definitions • Lower bounds: Relaxations • Lower bounds: Subgradient optimization • Approximation algorithms

49. In each step, the set of all possible solutions is split into two or more subsets, which are represented by branches in a decision tree. For the TSP, an obvious criterion for dividing all tours into subsets is whether they contain a given edge or not. The approach becomes really useful only when we add an extra idea: in each step, we will calculate lower bounds for the weight of all solutions in the respective subtrees and compare them with some previously computed upper bound. Then no branch of the tree for which the lower bound exceeds the known upper bound can possibly lead to an optimal tour.

50. Let W = (wij) be the weight matrix of a given TSP on Kn. We choose the diagonal entries wii as ∞ ; this can be interpreted as forbidding the use of loops. Let us select some row or column of W, and let us subtract a positive number d (smallest entry in our row or column) from all its entries. So that the weight of π with respect to W’ is reduced by d compared to its weight with respect to W. In particular, the optimal tours for W coincide with those for W’. We continue this process until we obtain a matrix W” having at least one entry 0 in each row and each column; such a matrix is called reduced. Note that the optimal tours for W agree with those for W”, and that the weight of each tour with respect to W” is reduced by s compared to its weight with respect to W, where s is the sum of all the numbers subtracted during the reduction process. It follows that s is a lower bound for the weight of all tours.