1 / 29

David Gao Alex Rubinov Prof. of Mathematics, Federation University

MINLP 2014, Carnegie Mellon, June 2-5, 2014. Canonical Duality Theory for Solving General Mixed Integer Nonlinear Programming Problems with Applications. David Gao Alex Rubinov Prof. of Mathematics, Federation University Research Prof. of Eng. Science, Australian National University.

yuri
Download Presentation

David Gao Alex Rubinov Prof. of Mathematics, Federation University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MINLP 2014, Carnegie Mellon, June 2-5, 2014 Canonical Duality Theory for Solving General Mixed Integer Nonlinear Programming Problems with Applications David Gao Alex Rubinov Prof. of Mathematics,Federation University Research Prof.of Eng. Science, Australian National University 1. Duality Gap between Math and Physics  conceptual problems 2. Canonical Duality-Triality: Unified Modeling Unified Solutions 3. Challenges  Breakthrough Supported by US Air Force AFOSR grants Since 2008

  2. Nonlinear/Global Optimization Problem: min f (x) s.t. g(x) ≤ 0 Gap between Math and Mechanics f(x) is an “objective” function g(x) is a general constraint. (naive) questions: What is the objective function? target and cost? what is Lagrangian? … “Mathematics is a part of physics. …In the middle of the twentieth century it was attempted to divide physics and mathematics. The consequences turned out to be catastrophic." — V.I. Arnold (1997) Mathematics needs to remarry physics – A. Jaffe Gao-Ogden-Ratiu, Springer Duality in mathematics is not a theorem, but a “principle” – Sir M.F. Atiyah Duality gap is not allowed in mathematical physics!

  3. Pd P max =max s min= min x min=max Canonical Duality-Triality Theory Gao-Strang, 1989 MIT and Gao, 1991 Harvard  A methodological theory comprises mainly • Canonical dual transformation Unified Modeling 2. Complementary-Dual Principle Unified Solution 3. Triality Theory Identify both global and local extrema Design powerful algorithms Unified understanding complexities Nothing is too wonderful to be true, if it be consistent with the laws of nature Michael Farady (1860 AC)

  4. Philosophical Foundation I-Ching (易 經 2800 BC-2737 BC): The fundamental Law of Nature is the Dao : the complementarity of one yin (Ying) and one Yang 一 陰 一 阳 以 谓 道 Laozi: All things have the receptivity of the yin and the activity of the yang. Through union with the life-giving force (chi) they blend in harmony Everything = {( Yin, Yang) ; Chi } = { (subj. , obj.) ; verb } Canonical System = { (Ying, Yang) | H-Chi } = { ( X , X* ) | A }

  5. P x out put x input f Convex Canonical System: Unified Modeling Xax ∂F( x) Convex System x* = f Xa* D D* = DT (P): minP(x) = W(Dx) - F( x) s.t. x  Xc= { xXa | Dx Ya} Yay y* Ya* ∂W(y) F(x) = f T x Subjective function The 1st duality: x*=∂F( x) = f , action-reaction W( y ) :Objective function (Gao, 2000): W( Q y ) = W( y )  QT = Q -1, det Q = 1 Exam: W(y) = ½ | y |2 , |Q y|2 = yTQTQ y = | y|2 The 2ndduality: y* = ∂ W(y) Constitutive law Pd Legendre transf. W*( y*) = yT y - W(y) min P(x) = maxPd(y*) Lagrangian: L(x, y*) = (Dx)T y* - W*( y*) - f T x = xT ( DTy* - f )- W*( y*) (Pd ): max Pd(y*) = - W*( y*) s.t. DT y* = f frame-indifference

  6. Objectivity is not a hypothesis, but a principle. P.G. Ciarlet, Nonlinear Functional Analysis, 2013, SIAM Objectivity, Gao 2000

  7. D* D Manufacturing Company System F( x ) = xT x* Xa Products x Price x*  Xa* Company Ya  Ya* Workers y Salary y* (P): min P(x) = W(Dx) – F( x ) income cost Target (Lose)

  8.  u= g(y) u*= g*(y*) 0 ≤ u u ┴u* u ┴ u* u* ≤ 0 0 ≥ u u* ≥ 0 Xa (x, x*) Xa*  u= Bx u*=B*x* Dmn D*=DT Ya ( y; y*) Ya* (P): min P(x) = W(Dx) – U( x) W(y) : Ya = { yY | g(y) ≥ 0 } physically feasible U(x) : Xa = { xX | Bx ≤ 0 } geometrically feasible Unified Understanding Constraints (Gao, 1997) Boundary (external) constraints in Xa  external KKT conditions 0 ≥ Bx = u ┴ u* = B*x* ≥ 0 Constitutive (objective) constraints in Ya  internal KKT conditions 0 ≤ g(y) = u┴u* = g*(y*) ≤ 0 Indicator ( J-J Moreau, 1963) W (y) = { ∂W constitutive law and KKT conditions W(y)ifg(y) ≥ 0 ∞otherwise Math = { ( X, X* ) ; A} = Obj. – Subj. (P): min P(x) = W(Dx) – U( x) , xX

  9. e* = ∂V y= ∂W e y Nonconvex W(y) (P): minP(x) = W(Dx) – x T f Canonical Duality - Triality Theory 1. Canonical transf. choose an objective measure e =Lx) W(D x) = V(L(x)) convex in e  canonical dual eqn (one-to-one): s ∂ V (e) Legendre Trans: V*(se Ts –Ve) Total complementary function (Gao-Strang, 1989) X(x, s ) = Lx) Ts- V*(s ) – x T f L D x* x (Quadratic L) = ½ x T G(s )x - V*(s ) – x T f ∂xX = 0 Analytic solution: x = Gs -1f Lt* D* Canonical Dual: Pd(s ) X (x (s), s ) = - ½ f T Gs -1f - V*(s  2. Complemenary-Dual Principle: Gap function e y e* y* If sc is a critical point of Pd(s ), then xc = Gsc  -1f is a critical solution of (P)and P(xc ) =Pd sc Let S+ = {s | G s  0 } G-Strang (1989) If sc  S+, then P(xc ) = min P(x) = max Pd(s) = Pd(sc ) 3. Triality Theory: S- = {s | G s < 0 } If sc S-, then either P(xc ) = max P(x) = max Pd(s) = Pd(sc ) (Gao, 1996) or P(xc ) = min P(x) = min Pd(s) = Pd(sc )

  10. W(y) = ½ ( ½ y 2 - 1)2 Pd  s y f P s x P(x) = W(Dx) – F(x) = ½( ½ |x|2 - 1 )2– xT f Example: Nonconvex in Rn Convex in R1 e =½ |x|2  V(e) = ½ (e – 1 )2 P s = ∂ V(e ) = e - 1 n=1: double-well Pd(s) = -½ | f |2s-1- ½s2- s ∂Pd(s) = 0 s2 (s+ 1) = ½ | f |2 s3 ≤s2≤ 0 ≤s1 x Pd Complementary-Dual Principle: Analytic solutions: xk = (s k) -1 f P(xk) = Pd(sk) k =1,2,3 Triality Theory: P(x1 ) = Pd(s1 )  P(x2 ) = Pd(s2 )  P(x3 ) = Pd(s3) Buridan’s donkey Open Problem (2003): If dim x ≠ dim s P(x2 ) = minP(x) ≠ mins<0 Pd(s ) = Pd(s2 ) 4 f = 0 Multiple solution Solved in 2012 Perturbation: f ≠ 0 Unique solution n=2: Mexican hat

  11. (P): minP(x) = ½ xTAx – f T x s.t. x {-1,1}n Quadratic Boolean Programming • Canonical transformation: e i = x i 2 – 1 ≤ 0 • (x, s  P(x) + S si  xi 2 - 1 ) = ½ x TG(s ) x - S si- f T x, • G (s ) = A+2 Diag (s ) xX (x, s) = 0 x = G(s ) -1 f (Pd):maxPd(s) = - ½ f T [G(s) ]-1 f – S si s.t.s S + = {s Rn | s ≥ 0, G(s)  0 } min =min P(x) KKT: si ≥ 0 , ei = xi2 - 1 ≤0, ( xi2 -1 )si = 0 si≠ 0 xi2 =1 integer! minP(x)=max Pd(s) Thm (Gao,2007): For each critical point sc ≠ 0 , the vector xc = G -1(sc)f {-1,1}n is a KKT point of P(x) and P(xc ) = Pd(sc ) if G(sc)  0P(xc )= min P(x) = max Pd (s) =Pd (sc ) if G(sc)  0 P(xc )= min P(x) = min Pd (s) =Pd (sc ) (P) Could be NP-Hard if Pd (s) has no critical point in S +

  12. Results for Max-Cut Problem (NP-Complete) Wang-Fang-Gao-Xing (2012) J. Global Optimization maxP(x) = ½ xTAx s.tx {0,1}n – f T x linear perturbation (Pd):maxPd(s) = - ½ f T [G(s) ]-1 f – S sis.t.G(s)≥ 0 Comparison of the running time produced by the canonical dual approach and GW’s approach (Goemans and Williamson)

  13. Max -Cut Problem (contin.) ■ Randomly produce 50 instances on graphs of sizes 20,50, 100, 150,200 and 500. The weight of each edge is uniformly from [0,10] ■ Ave ratio is the average approximate ratio, the ratio is close to 1 when the dimension increases

  14. The 2nd Canonical Dual for Integer Programming (P):min P(x) = ½ xTAx – f T x s.t. x {-1,1}n The second canonical dual (Gao, 2009) (Pg): min Pg(s) = - ½ sT A-1s – S | fi - si | s.t s Rn P(x) • Nonconvex/nonsmooth minimization • DIRECT method (Deterministic ) Pg(s) Thm: If scis a solution of ( Pg) , then xc i= { is a feasible solution of (P) and P(xc ) = Pg(sc) . • if fi > sc i • -1 if fi < sc i P(x) Pg(s) If A 0, P(xc )= minP(x)= maxPg(s)= Pg(sc) If A 0, P(xc )= min P(x)= min Pg(s) = Pg(sc) If A = - B T B , BRm n , Pg(s) = ½ sTs – S | fi - Bjisj | m < n

  15. n.m

  16. General MINLP Problems (P): min P(x,y ) = W(x,y) + aT x – bT y , x Xa , y Ya s.t.C1x + C2 y ≤ c , D1x + D2 y = d , Xa = {x Rn | 0  x  u }, Ya = { yZm | 0  y  v } • Let z = (x, y) , assume W(z ) is objective such that • an objective measure e =Lz ) and a convex V(e )  • W(z ) = V(L(z )) Canonical form: min P(z ) = V(L(z )) – f T z s.t.z  Za

  17. Mixed Integer (fixed Cost) Problem(with H.D. Sherali and N. Ruan) (P): min P(x,y) = ½ xTA x + cT x – f T y s.t. -y≤ x ≤ y, y{ 0 , 1 }n (Pd): maxPd(s ) = - ½ cTG(s )-1c - ½ S(si - fi )+ s.t. s ≥ 0 , G(s )= A +2 Diag (s ) p.d. Thm: If scis a solution of (Pd ) , then xc = - G (sc)-1 c , yci = { is a global solution of (P) and P(xc ,yc ) = Pd(sc ) 1 if fi < sc i 0 if fi > sc i Applications to scheduling and decision science x Rd x n

  18. Problems that can be solved Benchmark Problems: Rosenbrock function Lennard-Jones potential minimization Three Hump Camel Back Problem Goldstein-Price Problem 2n order polynomials minimizations Canonical functions … New math– Nonlinear space

  19. Nonconvex constrained problems (P): minP(x) = || y – z || 2 s.t.h(y) = ½ y A y – r ellipsoid g (z) = ½ ( || z – c || 2 - b )2 – d t ( z - c) Lagrangian: x = ( y, z ) R2n L(x, l, m ) = || y – z || 2 + l h(y) + m g(z) Let e = Lz ) = || z – c || 2 , V(e) = ½ (e - b ) 2 • s = ∂V(e) = e - b , V*(s ) = e s - V(e) =½ s 2 + bs Total complementary function X (x, l , m, s ) = || y – z || 2+ l h(y) +m [ L(z )s - V* (s) – d t ( z - c) ] G(l,m,s) =  0 (Pd):Pd(s ) = minxX (x, l , s ) = - ½ F T G(l,m,s) -1 F - mV* (s ) y z Thm: If G(l, m, s )  0 , (Pd)has at least one critical solution which gives to a global optimal solution to (P).

  20. Since 2010, Zalinescu (+ 2) has wrote 11 papers + 1 letter challenging the Canonical Duality Theory, which can be grouped in three categories: Challenges  Super-Duality 1. Conceptual Duality (4 papers, two published and two rejected) • min P(x) = V(L(x)) – F(x) • F (x) external energy (must be linear function)  ∂F(x) = x* = f • V(e ) internal (stored) energy (must be objective )  ∂V(e ) = s 2. Moral Duality (6 papers) all on the same open problem left in 2003: If dim P ≠ dim Pd minP(x) ≠ minPd(s ) s  S- 3. Multi-scaleduality (1 paper): Locally correct but globally wrong Certain condition in S+ is missing Total complementary function X (x, l , m, s ) , x = ( y , z )  R2n  0 y z

  21. “Counter-Example”  Hidden truth Conclusion: The consideration of the Gao-Strang function X (x, l , m, s )is useless, at least for the problem studied in [3]. Morales-Gao (2012): linear perturbation X (x, l , m, s ) – k -1 xT f

  22. Graph, lattice, fuzzy max-plus algebra Mixed Integer Optim. Supply Chain Process FEM, FDM, FVM, SDP Meshless, Wavelet, SIP Nonconvex/nonsmooth Variational/V.I. Analysis Unified Global Optimization Discrete optimization Combinatorial Algebra Combinatorial Optim. Integer Programming Canonical Duality-Triality Theory Numerical Analysis Continuous Optimization

  23. Duality in Nonconvex Systems:Theory, Methods and ApplicationDavid Yang GaoKluwer Academic Publishers, 2000, 454pp • Part I    Symmetry in Convex Systems • 1.   Mono-duality in static systems • 2.   Bi-duality in dynamical systems • Part II    Symmetry Breaking: • Triality Theory in Nonconvex Systems • 3.   Tri-duality in nonconvex systems • 4.   Multi-duality and classifications of • general systems • Part III    Duality in Canonical Systems • 5.   Duality in geometrically linear systems • 6.   Duality in finite deformation systems • 7.   Applications, open problems and concluding remarks • duality in fluid mechanics ?

  24. All happy families are alike, Every unhappy family is unhappy in its own way Anna Karenina --- Leo N Tolstoy Reason: canonical duality Reason: different duality gaps Philosophy = Love of Canonical Duality Proof: 1. By Greeks: Philosophy = Love of Wisdom 2. By Confucius: The highest Wisdom = Dao 3. By I-Ching (4000BC): Dao = one Ying + one Yang = Canonical Duality 一 陰 一 阳 以 谓 道 --- 易 經 Open Problem: How to correctly understand the Triality

  25. Rn xx* Rn  Rm nL Lyf yy* Rm Rm LL LRmr Rr Rr oxxo u Canonical Duality –Triality Theory: 1. Non-convex  concave 2. Discrete  continuous 3. Non-smooth  smooth 4. Rescaling: RnRm Rr n > m > r Thanks! 5. Diff. eqn  Algebraic eqn. L*L x  = 0 6. Non-deterministic  deterministic 7. Challenges Breakthrough Open Problems: (P) is NP-Hard if (Pd) has no solution in Sa+ ? The 4thWorld Congress on Global Optimization Gainesville, Florida - USA, Feb 22-25, 2015

  26. Some references [1] Gao, D.Y. and Sherali, H.D. (2008). Canonical duality: Connection between nonconvex mechanics and global optimization, in Advances in Appl. Mathematics and Global Optimization, 249-316, Springer, 2008 [2] Gao, D.Y. (2009). Canonical duality theory: Unified understanding and generalized solution for global optimization problems, Computers & Chemical Engineering, 33:1964–1972 [3] Daniel Morales-Silva, David Gao On the minimal distance between two surfaces, http://arxiv.org/abs/1210.1618 [4] Gao, DY and Wu, C, On the Triality Theory in Global Optimization http://arxiv.org/abs/1104.2970

  27. The 4thWorld Congress on Global Optimization Gainesville, Florida - USA, Feb 22-25, 2015 International Society of Global Optimization (www.iSoGOp.org) Thanks! WCGO 2015

More Related