1 / 40

The Limit of Polynomials

This paper explores the limitations and extensions of polynomials in program optimization, focusing on affine scheduling, Farkas’ lemma, polynomial scheduling, and implications for scheduling. It also discusses the differences between Farkas’ lemma and Handelman’s theorem and their effects on scheduling. The paper presents examples and discusses the impact of these findings on polynomial optimization.

ajackson
Download Presentation

The Limit of Polynomials

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Limit of Polynomials Tomofumi YukiINRIA Rennes

  2. Acknowledgements • Interesting discussions about polynomials • Paul Feautrier • Steven Derrien • Silviu-IoanFilip • Year-Long Student Project • AdrienChaffangeon • AdrienGougeon • TimotheeAnne IMPACT 2019

  3. Limits of the Polyhedral Model • Strong framework for: • dependence analysis • program transformation • code generation • ... • Strong limitation in exchange: • everything must be affine IMPACT 2019

  4. Polynomial Extensions • Extend the class of functions (Feautrier2015) • affine to polynomial • Lift restrictions of the polyhedral model: • polynomial scheduling • non-linear array accesses (e.g., A[i*N+j]) • parametric tiling • + some work on code gen (IMPACT 2018) IMPACT 2019

  5. Affine Scheduling • Ordering of operations • assign time stamps (affine functions) • model of loop transformations • dependences must be respected for (i = 1:N) for (j = 1:N) S1(i,j); for (i = 1:N) S2(i); for (i = 1:N) for (j = 1:N) S1(i,j); S2(i); Domain S1:{i,j|1≤i≤N, 1≤j≤N} S2:{i|1≤i≤N} Schedule sch1(i,j) = (i,j) sch2(i) = (i,N+1) IMPACT 2019

  6. Farkas’ Lemma • Given polyhedral set D and affine function f • f is non-negative over D • ifff can be expressed as a linear combination of constraints defining D • Scheduling ≈ Positivity Check • op1 and op2 are defined with polyhedral sets sch1(op1) > sch2(op2) f(op1,op2) > 0 IMPACT 2019

  7. Polynomial Scheduling • Handelman’s Theorem (simplified) • Given a polynomial f, and a set • f is strictly positive on D • iff it has the following representation Handelman Representation: linear combination of products of constraints IMPACT 2019

  8. Two Research Questions • When do we need polynomial scheduling? • multi-dimensional affine is quite powerful • static but non-affine programs? • What are the implications of replacingFarkas’ Lemma with Handelman’s Theorem? • non-negative / strictly positive • unbounded D / bounded D • No bound on degree of constraint products Index Set Splitting? IMPACT 2019

  9. Implications for Scheduling • A class of polynomials does not have exact Handelman representation (Lasserre 2002) • Polynomial Scheduling Caveats: • a class of schedules is not explored • effectively acts as non-negativity certificate • constant matters Jean B. Lasserre. 2002. SemidefiniteProgramming vs. LP Relaxations for Polynomial Programming. Mathematics of Operations Research IMPACT 2019

  10. Outline • Introduction • FarkasvsHandelman • Polynomial Optimization • Lasserre’s Results • Parametric Domains • Conclusion IMPACT 2019

  11. Ex1: Farkas’ Lemma • D: x∈[-1,1] • x+1≥0 • 1-x≥0 • Test if 2x+1 is positive in D • Can 2x+1 be expressed as • a(x+1)+b(1-x)+c • where a,b,c≥0? • No: 2x+1 is not non-negative in D 2x+1 -1 1 IMPACT 2019

  12. Ex2: Handelman’s Theorem • D: x∈[-1,1] • x+1≥0 • 1-x≥0 • Test if x2+1 is positive in D • Consider degree 2 products: [1,x+1,1-x,(x+1)2,(1-x)2,(x+1)(1-x)] • Can x2+1 be expressed with above? • Yes: (x+1)2+(1-x)2 = 2x2+2 • 0.5(x+1)2+0.5(1-x)2 = x2+1 x2+1 -1 1 IMPACT 2019

  13. Key Differences • The set D must be compact • not the case for Farkas’ Lemma • Potentially infinitely many terms • products of constraints introduce more terms • in practice: bound the degree (Σki) • Strict positivity • representation should give non-negativity IMPACT 2019

  14. Ex3: Degree Bound on Products • Same test with different D • D: x∈[-2,2] • x+2≥0 • 2-x≥0 • Consider degree 2 products: [1,x+2,2-x,(x+2)2,(2-x)2,(x+2)(2-x)] • Can x2+1 be expressed with above? • No: need degree 5 products x2+1 -2 2 IMPACT 2019

  15. Ex4: Strict Positivity • Test for x2instead • D: x∈[-1,1] • x+1≥0 • 1-x≥0 • Consider degree 2 products: [1,x+1,1-x,(x+1)2,(1-x)2,(x+1)(1-x)] • Can x2 be expressed with above? • No: you cannot express x2 • even with high degree products x2 -1 1 IMPACT 2019

  16. How is Scheduling Affected? • Polynomial Scheduling seems to “work” • but not the same as Farkas Scheduling • the impact of differences is unclear • Some answers from Polynomial Optimization • main results by Jean-Bernard Lasserre IMPACT 2019

  17. Outline • Introduction • FarkasvsHandelman • Polynomial Optimization • Lasserre’s Results • Parametric Domains • Conclusion IMPACT 2019

  18. Polynomial Optimization • Find the minimal value of a polynomial over a domain • links to positivity checks • Recall: positivity checks characterize positive functions over a domain z -1 1 IMPACT 2019

  19. Ex5: Finding the Minimum Value • D: x∈[-1,1] • Find: min x-x2 • Consider degree 2 products: 1 x+1 1-x (x+1)2 (1-x)2 (x+1)(1-x) x-x2 -1 1 IMPACT 2019

  20. Ex5: Finding the Minimum Value • D: x∈[-1,1] • Find: min x-x2 • Their linear combination: λ0 + λ1(x+1) + λ2(1-x) + λ3(x+1)2 + λ4(1-x)2 + λ5(x+1)(1-x) x-x2 -1 1 IMPACT 2019

  21. Ex5: Finding the Minimum Value • D: x∈[-1,1] • Find: min x-x2 • Expand the squares: λ0 + λ1(x+1) + λ2(1-x) + λ3(x2+2x+1) + λ4(x2-2x+1) + λ5(-x2+1) x-x2 -1 1 IMPACT 2019

  22. Ex5: Finding the Minimum Value • D: x∈[-1,1] • Find: min x-x2 • Factor by monomials: 1(λ0+λ1+λ2+λ3+λ4+λ5) + x(λ1-λ2+2λ3-2λ4) + x2(λ3+λ4-λ5) x-x2 -1 1 min λ1=λ5=1 =1 =-1 x-x2≥-2 IMPACT 2019

  23. Ex6: Finding the Minimum #2 x2-x • Find: min x2-x • Minimum is -0.25 • Solution with different degrees: -1 1 fixed degree  lower bound(relaxation to LP) NEVER reaches -0.25!! IMPACT 2019

  24. Lasserre’s Theorem • When exact solutions can be found • Theorem 3.1 (Lasserre 2002) • M: bound on degree of constraint products • For a class of polynomials, solution to the relaxed problem approaches the exact solution as M→∞ (i.e., never reached) • The class in question: • When a global minimizer is at the interior of D IMPACT 2019

  25. Back to Ex5 and Ex6 x2-x • Ex6 has its minimizer at the interior of D Ex5 Ex6 x-x2 -1 1 -1 1 minimizers IMPACT 2019

  26. What Does it Mean? • A class of polynomials cannot be found with polynomial scheduling • Example: • x2 for x∈[-1,1] • x2+1 needs M=2 • x2+0.25 needs M=5 • x2 needs M=∞ • x2 can never be found! • expressing x2+c  harder as c→0 x2 -1 1 IMPACT 2019

  27. Relation to Strict Positivity • Lasserre’s Theorem shows when strict positivity in Handelman’s Theorem manifest • comes from M→∞ • The following are equivalent: • both require minimizers to be at the boundary Exact Solution to Polynomial Optimization Non-Negativity Certificate with Handelman IMPACT 2019

  28. Back to Ex5 and Ex6 Again x2-x • Non-Negativity Certificate only for Ex5 Ex5 Ex6 x-x2 -1 1 -1 1 x2-x+0.25+ε≥0 x-x2+2≥0 x2-x+0.25>0 IMPACT 2019

  29. What it Means for Scheduling • If Handelman Representation can be found with some M depends a lot on the polynomial • exploration space is sparse • Some are not expressible: • non-negative polynomials with global minimizer at interior • Constant Matters: • smaller constant need higher M IMPACT 2019

  30. Optimization vs Scheduling • One subtle difference: • no constants in optimization context • Minimizer can be at the interior for scheduling x2+1 x2+1 found with M=2 x2≥-1 is not exact LB of x2 -1 1 need “sufficiently high” M depending on the constant IMPACT 2019

  31. Outline • Introduction • FarkasvsHandelman • Polynomial Optimization • Lasserre’s Results • Parametric Domains • Conclusion IMPACT 2019

  32. On Parametric Domains • Everything so far: when set D is compact • matches hypotheses of Handelman’s Theorem • We use parametric domains all the time • usually not compact • Can we still use Handelman’s Theorem? IMPACT 2019

  33. Problem with Parameters • D: x∈[-N,N]; Find: x2+1 • family of D[-1,1], [-2,2],... • Every instance can bescaled to [-1,1] • Harder problem with higher values of N • N→∞ ≈ find x2 for x∈[-1,1] x2+1 x2+0.25 -2 -1 2 1 No hope of finding x2+1 for x∈[-N,N] IMPACT 2019

  34. Parametric Solutions are Possible • D: x∈[-N,N]; Find: x2+N2 • All instances are equivalent: • x2+N2can be found with M=2 • (x+N)2+(N-x)2 = 2x2+2N2 • solution for N=1 can be made parametric x2+1 x2+4 x2+9 -1 -2 -3 3 1 2 IMPACT 2019

  35. Impact of Parameters • Further reduce the space of polynomials • the same linear combination must work for all parameter instances • need monomials involving parameters • Example: IMPACT 2019

  36. Conclusion • Closer look into what it means to use Handelman’s Theorem for scheduling • a good chunk of the space is excluded • especially with parameters • can be treated as non-negativity certificate • Completely skipped ISS part! • led me to try and find polynomials with global minimizers at interior IMPACT 2019

  37. Semi-Definite Programming? • SDP relxation • another approach to Polynomial Optimization • sum of squares instead of linear combination • Lasserre’s paper was about LP vs SDP • SDP is better! • Can we use SDP approach for scheduling? • seems promising • but compactness hypotheses remain IMPACT 2019

  38. IMPACT 2019

  39. Index-Set Splitting • Dependence Graph • no affine schedule • need PW-affine • Main reason: • affine function cannot change“direction” IMPACT 2019

  40. Polynomial Guided ISS • Polynomials are multi-directional: • e.g., x2 • Main idea: • first find apolynomial schedule • analyze polynomial • infer necessary pieces • Example: • (i-j)2 split at i=j j i IMPACT 2019

More Related