1 / 34

Discrete Optimization via Branch and Bound with Soft Arc Consistency

Discrete Optimization via Branch and Bound with Soft Arc Consistency. William Lam March 20, 2013 (with slides from the IJCAI-09 tutorial on Combinatorial Optimization for Graphical Models). Introduction. Bayes nets/Markov random fields/ f actor graphs

boaz
Download Presentation

Discrete Optimization via Branch and Bound with Soft Arc Consistency

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete Optimization via Branch and Bound with Soft Arc Consistency William Lam March 20, 2013 (with slides from the IJCAI-09 tutorial on Combinatorial Optimization for Graphical Models)

  2. Introduction • Bayes nets/Markov random fields/factor graphs • Most Probable Explanation (MPE), Maximum a Posteriori (MAP): maximizing probability • Weighted CSP/Cost function network • Minimum cost • Problems are reducible to each other

  3. Search Basic Step: Conditioning • Select a variable X1 X2 X3 X4 X5

  4. Search Basic Step: Conditioning X1 X2 X3 X4 X5 …... X1  a X1  c X1  b X2 X2 X2 X3 X3 X3 …... X4 X5 X4 X5 X4 X5

  5. Depth-First Branch and Bound (DFBB) Each node is a subproblem (defined by current conditioning) Variables (LB) Lower Bound = f under estimation of the best solution in the sub-tree f k If  UB then prune LB = k (UB) Upper Bound = best solution so far

  6. Computing Bounds • Static Mini-Bucket heuristics (Kask & Dechter, AIJ’01), (Marinescu & Dechter, IJCAI’05) • (with max-marginal matching) (Ihler et al. UAI’12) • Dynamic Mini-Bucket heuristics (Marinescu & Dechter, IJCAI’05) • Maintaining local consistency (Larrosa & Schiex, AAAI’03), (de Givry et al., IJCAI’05)

  7. Local Consistency in Constraint Networks • Massive local inference • Time efficient (local inference, as mini buckets) • Inferonlysmallconstraints, added to the network • No variable iseliminated • Produces an equivalent more explicit problem • May detectinconsistency (prune treesearch) Arc consistency inference in the scope of 1 constraint

  8. Arc Consistency (binary CSP) • for a constraintcAB and variable A cAB+ cB v v  0 0 ElimB(min) 0  0 w w  A B • Appliediteratively on all constraint/variables • Emptydomain => inconsistency

  9. Arc Consistency and CostFunctions • for a costfunctionfABand a variable A fAB+ fB v v 2 0 1 Elim B(min) 1 0 w w 0 1 A B EQUIVALENCE LOST

  10. ShiftingCosts (cost compensation) Subtract from source in order to preserve the problemEquivalence Preserving Transformation v v 1 2 0 1 Elim B 1 0 0 w w 1 A B 1 2 1 0

  11. Complete Inference vs Local Inference • Complete inference • Combine, eliminate, add & forget • Systematicinference • Exponential time/space • Preserves optimum • Provides the optimum f • Local consistency • Combine, eliminate, add & subtract • Massive local inference • Space/time efficient • Preservesequivalence • Provides a lb f

  12. Equivalence Preserving Transformation • ShiftingcostsfromfABto A Shift(fAB,(A,w),1) • Can bereversed(e.g. Shift(fAB,(A,w), -1)) v v 1 0 1 0 0 w w 1 A B Arc EPT: shift cost in the scope of 1 cost function Problem structure preserved

  13. Shift(fAB,(A,b),1) Shift(fAB,(B,a),1) A B A B Shift(fA,(),1) Shift(fAB,(B,a),-1) Equivalence Preserving Transformations A B • EPTsmay cycle • EPTsmay lead to different f0 • WhichEPTsshouldweapply?

  14. Node Consistency (NC*) (Larrosa, AAAI 2002) • For any variable A • a, f+fA(a)<k •  a, fA (a)= 0 • Complexity: O(nd) k = 4 f= 0 1 A v 3 2 C w 0 2 v 0 1 1 0 1 w v 0 1 1 w 1 B Shift(fC, ,1) Shift(fA, ,-1);Shift(fA, ,1)

  15. Arc Consistency (AC*) (Schiex, CP 2000) (Larrosa, AAAI 2002) • NC* • ForallfAB • a b fAB(a,b)= 0 • bis a simple-support • complexity: O(n 2d 3) k=4 f= 1 2 A C w 0 2 v 0 1 0 w v 0 1 1 0 1 0 w 1 B Shift(fB,,1) Shift(fC,,-2) Shift(fC,,2) Shift(fAC,(C,v),2) Shift(fBC,(B,v),1) Shift(fBC,(B,w),1)

  16. Full Arc Consistency (FAC*) • NC* • ForallfAB • a b fAB(a,b) +fB(b) = 0 • bis a full-support • Too strong to enforce

  17. Directional AC (DAC*) (Cooper, Fuzzy Sets and Systems 2003) • NC* • ForallfAB (A<B) • a b fAB(a,b) +fB(b) = 0 • complexity: O(ed2) A<B<C k=4 f= 2 1 A v 2 2 C w 0 2 v 0 1 1 0 w 1 v 0 1 1 1 w 0 1 B Shift(fB, ,1) Shift(fBC,(C,v),-1) Shift(fBC,(B,v),1) Shift(fBC,(B,w),1) Shift(fA,, -2) Shift(fA,, 2)

  18. OtherLocal Consistencies • FDAC* = DAC+AC+NC • Strongerlowerbound • O(end3) • Bettercompromise • EDAC* = FDAC+ EAC (existential AC) • Evenstronger • O(ed2max{nd, k}) • Currentlyamongthebestpracticalchoice (Cooper, Fuzzy Sets and Systems 2003) (Larrosa & Schiex, IJCAI 2003) (Larrosa & Schiex, AI 2004) (Cooper & Schiex, AI 2004) (de Givryet al., IJCAI 2005) (Sanchez et al, Constraints 2008)

  19. FDAC* • DAC* + AC* • ForallfAB (A<B) • a b • fAB(a,b) +fB(b) = 0 • ForallfAB(A>B) • a b • fAB(a,b)= 0 A<B<C k=4 f= 2 A C w 0 v 2 0 2 0 w v 0 w 0 B

  20. Existential Arc Consistency (EAC*) • Gettingcloserto FAC* • ForallfAB • a b • fA(a) = 0 • fAB(a,b) +fB(b) = 0 • (A full support in both directions) • If a variable A is not EAC, we can enforce full-supports and break NC • Leads to increasing lower bound

  21. EDAC* • FDAC* + EAC* • Every value is fully supported in one direction and simply supported in the other (FDAC*) • At least one value per variable is fully supported in both directions (EAC*) • FDAC* < EDAC* < FAC*

  22. FDAC*

  23. EDAC*

  24. EDAC* EAC* DAC* AC* NC*

  25. Other (Stronger) Consistency Properties • Optimal Soft Arc Consistency • Finds an optimal set of rational EPTs (Cooper et al., IJCAI’07) (Schlesinger, Kibernetika 1976) • Virtual Arc Consistency • Finds an improving sequence of rational EPTs (Cooper et al., AAAI’08)

  26. Higher Arity Functions • Consistency enforcing ignores functions with arity > 2 • Must wait until sufficient conditioning reduces function arity to 2 • Enhancement: preprocess problem before search • Perform EPTs on functions with arity > 2 to generate binary functions

  27. Variable/Value Selection • Weighted Degree • For each constraint, count the number of times an inconsistency is reached during search • Variable xi weight is the sum of the weights of all constraints involving xiand another uninstantiated variable (Boussemartet al., ECAI’04) • Last conflicts • Maintain a set of variables that resulted in a dead end and prioritize testing these first (Lecoutre et al., AI’09) • Values: use unary functions as lower bounds

  28. Overall Algorithm • Input: a cost function network N (preprocessed to enforce consistency) • Output: solution cost • Initialize: UB = k solve(N): if all domains are empty, return N0 X = VariableSelectionHeuristic(N) a= argminN(X) N’ = EnforceConsistency(N|X=a) if N’0 >= UB, N’ = EnforceConsistency(N|X!=a) // remove a from N.D(X) return UB = min(UB, solve(N’))

  29. UAI’08 Competition Results

  30. UAI’08 Competition Results (II)

  31. UAI’08 Competition Results (III)

  32. Software • Toulbar2 http://mulcyber.toulouse.inra.fr/gf/project/toulbar2(Open source WCSP, MPE solver in C++) • Participated in CP06, CP08, UAI08, UAI10, and PIC11 competitions

  33. DAOOPT and Toulbar2 • DAOOPT (our AOBB-based solver) • Bounds computed before search via MBE • “Top-down approach”: Variable duplication to reduce cluster size • Re-parameterization to enforce consistency between copies of variables(FGLP+JGLP+MBE-MM) (Ihleret al., UAI’12) • Fast node expansion • Static variable ordering • Toulbar2 • Bounds incrementally improved during search via SAC • “Bottom-up approach”: Ignores larger factors until conditioning reduces their size • Re-parameterization directly generates bounds • Node expansion speed depends on consistency algorithm • Dynamic variable ordering

More Related