Discrete Optimization Lecture 5 – Part 1

1. Discrete OptimizationLecture 5 – Part 1 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://mpawankumar.info

2. Outline • Submodular Functions • Unary Submodular Functions • Pairwise Submodular Functions • Submodular Energy Function

3. Submodular Function Set S Function f over power set of S f(T) + f(U) ≥ f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

4. Supermodular Function Set S Function f over power set of S f(T) + f(U) ≤ f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

5. Modular Function Set S Function f over power set of S f(T) + f(U) = f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

6. Modular Function f(T) = ∑s ∈T w(s) + K Is f modular? YES All modular functions have above form? YES Prove at home

7. Diminishing Returns Define df(s|T) = f(T ∪{s}) - f(T) Gain by adding s to T If f is submodular, df(s|T)is non-increasing

8. Diminishing Returns Define df(s|T) = f(T ∪{s}) - f(T) Gain by adding s to T f(U ∪ {s}) + f(U ∪ {t}) ≥ f(U) + f(U ∪{s,t}) for all U⊆ S and distinct s,t ∈ S\U Necessary condition for submodularity Proof?

9. Diminishing Returns Define df(s|T) = f(T ∪{s}) - f(T) Gain by adding s to T f(U ∪ {s}) + f(U ∪ {t}) ≥ f(U) + f(U ∪{s,t}) for all U⊆ S and distinct s,t ∈ S\U Sufficient condition for submodularity Proof?

10. Proof Sketch Consider T, U ⊆ S We have to prove f(T) + f(U) ≥ f(T ∪ U) + f(T ∩ U) We will use mathematical induction on |TΔU|

11. Proof Sketch |TΔU| = 1 Either U ⊆ T or T ⊆ U Let T ⊆ U T ∪ U = U and T ∩ U = T Proof follows trivially

12. Proof Sketch |TΔU| = 2 If U ⊆ T or T ⊆ U, then proof follows trivially If not, then simply use the condition f(U ∪ {s}) + f(U ∪ {t}) ≥ f(U) + f(U ∪{s,t}) for all U⊆ S and distinct s,t ∈ S\U

13. Proof Sketch |TΔU| ≥ 3 Let t ∈ T\U Assume, wlog, |T \ U| ≥ 2 |T Δ ((T \{t}) ∪ U)| < |T Δ U| Why? f(T∪U) - f(T) ≤ f((T\{t}) ∪ U) - f(T\{t}) Induction assumption

14. Proof Sketch |TΔU| ≥ 3 Let t ∈ T\U Assume, wlog, |T \ U| ≥ 2 |(T\{t}) Δ U| < |T Δ U| Why? f((T\{t}) ∪ U) - f(T\{t}) ≤ f(U) - f(T ∩ U) Induction assumption

15. Proof Sketch |TΔU| ≥ 3 f(T∪U) - f(T) ≤ f(U) - f(T ∩ U) Hence Proved

16. Outline • Submodular Functions • Unary Submodular Functions • Pairwise Submodular Functions • Submodular Energy Function

17. Unary SubmodularFunction Set S = {1, 2, …, n} Consider a ∈S Function fa over power set of {a} fa(null set) = θa(0) fa({a}) = θa(1) Unary potentials Always When is fasubmodular?

18. Outline • Submodular Functions • Unary Submodular Functions • Pairwise Submodular Functions • Submodular Energy Function

19. Pairwise SubmodularFunction Set S = {1, 2, …, n} Consider a,b ∈S Function fab over power set of {a,b} fab(null set) = θab(0,0) fab({a}) = θab(1,0) fab({b}) = θab(0,1) fab({a,b}) = θab(1,1) Pairwise potentials When is fabsubmodular?

20. Pairwise SubmodularFunction Set S = {1, 2, …, n} Consider a,b ∈S Function fab over power set of {a,b} fab(null set) = θab(0,0) fab({a}) = θab(1,0) fab({b}) = θab(0,1) fab({a,b}) = θab(1,1) Pairwise potentials θab(0,0) + θab(1,1) ≤ θab(0,1)+ θab(1,0)

21. Outline • Submodular Functions • Unary Submodular Functions • Pairwise Submodular Functions • Submodular Energy Function

22. Energy Function Consider x ∈{0,1}n Set S = {1, 2, …, n} Energy function E(x) minx E(x) ∑aθa(xa) + ∑(a,b)θab(xa,xb) Assume θab(0,0) + θab(1,1) ≤ θab(0,1)+ θab(1,0) Submodular Energy Function