1 / 28

Practical Analysis of Non-Termination in Large Logic Programs

Practical Analysis of Non-Termination in Large Logic Programs. Senlin Liang and Michael Kifer. Motivation. High-level LP languages, e.g ., SILK http :// silk.semwebcentral.org Flora-2 http ://flora.sourceforge.net are designed to be suitable for knowledge engineers, who are

thanos
Download Presentation

Practical Analysis of Non-Termination in Large Logic Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Practical Analysis of Non-Termination in Large Logic Programs Senlin Liang and Michael Kifer

  2. Motivation • High-level LP languages, e.g., • SILK http://silk.semwebcentral.org • Flora-2 http://flora.sourceforge.net are designed to be suitable for knowledge engineers, who are notprogrammers • KBs created by engineers typically • are complex, large • stress the capabilities of the underlying engine =>Non-termination happens often – very hard to debug • To address this, we developed • Terminyzer – a non-Termination analyzer (this extends our previous work in PADL-13)

  3. Outline • Preliminaries: causes of non-termination, tabling, and forest logging • Adding ids to rules • Terminyzer: analyses causes, repairs problems • Experiments • Conclusion and future work

  4. Causes of Non-Termination • Cause 1: loops in SLD-resolution • Example p(X) :- p(X). ?- p(a). • Solution: tabling[SW12] • Caches calls to subgoals, which cuts recursive loops • If enough predicates are tabled then • Each subgoal is tabled once • Each answer is tabled once • Evaluation terminates if there are finitely many subgoals and answers [SW12] Terry Swift and David S. Warren. XSB: Extending prolog with tabled logic programming. TPLP’12.

  5. Causes of Non-Termination (cont’d) • Cause 2: • Engine supports tabling • But the program generates infinitely many tabled subgoals • Example p(X) :- p(f(X)). ?- p(a). Subgoals to be tabled: p(a), p(f(a)), p(f(f(a))), ... • Solution: subgoalabstraction [RS] • Abstracts subgoals that are deeper than a threshold • Assuming threshold = 2, p(f(f(f(a)))) would be abstracted to p(f(f(X))), X = f(a) • Guarantees that there will be only a finite number of tabled subgoals [RS] F. Riguzzi and T. Swift. Terminating evaluation of logic programs with finite three-valued models. ACM on Computational Logic. To appear.

  6. Causes of Non-Termination (cont’d) • Cause 3: • Engine supports both tabling and subgoal abstraction • But infinitely many answers • Example p(a). p(f(X)) :- p(X). ?- p(X). Answers to be derived: p(a), p(f(a)), p(f(f(a))), … • Solution: does not exist • Halting problem is undecidable • Whether a program has a finite number of answers is undecidable • We can only try to help the user to deal with the issue • If user really intended the program to be the way it is • We need a way to limit output. E.g., bounded rationality[BS13] • Our focus: Unexpected non-termination (ie, when it’s a bug) [BS13] B. Grosof and T. Swift. Radial restraint: A semantically clean approach to bounded rationality for logic programs. AAAI’13

  7. Tabling and Forest Logging • Tabling needs no introduction • Forest logging is a new tracing facility in XSB where • Timestamp preserves the order of events • status = new, complete, incomplete

  8. Adding IDs to Rules – Key Insight • Add unique ids to rules s.t. tabled subgoals remember their host rules: • Each tabling declaration :- table p/nis changed to :- table p/(n+1) • For a query ?- p(x1, …, xn) • If p/n is tabled, then • Change it to ?- p(x1, …, xn, Newvar) • Chop off the last arguments of returned answers • Otherwise, the query stays the same

  9. Terminyzer Overview • Two versions • Version 1 (most precise) requires: • Tabling • Forest logging • Subgoal abstraction • Version 2 requires only • Tabling • Forest logging • Currently only XSB has all three features, but: • Several systems have tabling • Forest logging info exists internally in all of them – just needs to be exposed to the user • => at least Version 2 is easily portable

  10. Terminyzer Overview • Suppose a query does not terminate, Terminyzer then • Analyzes an execution forest log • Determines the causes of non-termination • The exact sequence of unfinished tabled subgoals, and the host rule id of each subgoal • Subgoals forming recursive cycles • Rectifies some causes of misbehavior (heuristically)

  11. Call Sequence Analysis • Identifies the exact sequence of unfinished calls and their host rule ids that lead to a non-termination • Find unfinished subgoals– whose answers have not been completely derived – by unfinished(Child,Parent,Timestamp) :- tabled_call(Child,Parent,new,Timestamp), not_exists(completed(Child,SCCNum,Timestamp1)). • unfinished(child,parent,timestamp) says that • Subgoal parent calls subgoal child, and it happened at timestamp • Neither child nor parent have been completely evaluated • Sort unfinished calls by their timestamps • Host rule ids are kept in the last arguments of child-subgoals not_exists is the XSB well-founded negation operator; existentially quantifies SCCNum and Timestamp1.

  12. Call Sequence Analysis (cont’d) • Example 1 @!r1 p(a). @!r5 r(X) :- r(X). @!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X). @!r3 q(b). @!r7 s(f(b)). @!r4 q(g(X)) :- p(X). ?- r(X). where @!ruleidis the syntax to assign rule ids • Its unfinished calls – the red ones form a non-terminating loop unfinished(r(_h9900,_h9908), root, 0) – root is for intial query unfinished(r(_h9870,r5), r(_h9870,_h9889), 8) unfinished(r(_h9840,r5), r(_h9840,r5), 11) unfinished(p(_h9810,r6), r(_h9810,r5), 12) unfinished(q(_h9780,r2), p(_h9780,r6), 16) unfinished(p(_h9750,r4), q(_h9750,r2), 20) unfinished(q(_h9720,r2), p(_h9720,r4), 24) • Next, we will find recursive cycles

  13. Call Sequence Analysis (cont’d) • Unfinished calls can be represented using an unfinished-call graph UCG = (N,E) • N: the set of unfinished subgoals • E: {(parent,child) |unfinished(child,parent,ts) is true} • For the above example unfinished(r(_h9900,_h9908), root, 0) unfinished(r(_h9870,r5), r(_h9870,_h9889), 8) unfinished(r(_h9840,r5), r(_h9840,r5), 11) unfinished(p(_h9810,r6), r(_h9810,r5), 12) unfinished(q(_h9780,r2), p(_h9780,r6), 16) unfinished(p(_h9750,r4), q(_h9750,r2), 20) unfinished(q(_h9720,r2), p(_h9720,r4), 24) where: • Each node is represented by the timestamp when it is first called • -1 represents root • Edges are labeled with timestamps of calls • Loops in UCG represent recursive cycles • However, not all cycles are causing non-termination • E.g. [8, 8] Can be represented as thousands of subgoals

  14. Call Sequence Analysis (cont’d) • Assume all predicates are tabled + subgoal abstraction • Theorem(Soundness of the call sequence analysis) If there are unfinished calls in a query’s complete trace, then • Call sequence analysis finds the exact sequence of unfinished calls that caused non-termination, and • The ids of the rules that issued these calls • Theorem (Completenessof the call sequence analysis) If the evaluation of a query does not terminate, then • There is at least one loop in the UCG for its complete trace, and the loop’s subgoals are responsible for generating infinite number of answers, and • The last argument of each of these subgoalsspecifies the rule ids from whose bodies these subgoals were called. • The complete trace is infinite due to non-termination so, practically speaking • We work with only a prefix of the trace by limiting term depth/size or execution time/space • It may produce false negatives, but they are also useful for identifying computational bottlenecks.

  15. Answer Flow Analysis • Recall that not all cycles in UCG are causing non-termination, so we need to refine call sequence analysis • Answer flow analysis does precisely that: it identifies the cycles that actually cause non-termination • Non-termination happens if and only ifa subset of subgoalskeeps: • Receiving answers from producers, • Deriving new answers, and • Returning answers to callers • Answer flow analysis looks for repeated patterns of answer returns

  16. Answer Flow Analysis (cont’d) • Compute answer-flow patterns (AFP) • Answer-return sequence (ARS): the sequence of (child,parent) pairs where child returns answers to parent • Candidate AFP: a sequence cafps.t.cafp2+is a suffix of ARS • AFP: the shortest candidate AFP cafps.t. its repetition forms the maximal suffix of ARS among all candidate AFP’s • In previous Example 1 • ARS = [ (p(_h599,r4),q(_h599,r2)), (q(_h619,r2),p(_h619,r4)), (p(_h639,r4),q(_h639,r2)), (q(_h659,r2),p(_h659,r4)), (p(_h679,r4),q(_h679,r2)), (q(_h699,r2),p(_h699,r4)), (p(_h719,r4),q(_h719,r2)), (q(_h739,r2),p(_h739,r4)), (p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))]. where (child,parent) indicates child returns answers to parent • Candidate AFPs are: • cafp1 = [(p(_h759,r4),q(_h759,r2)), (q(_h779,r2),p(_h779,r4))] • cafp2 = cafp1• cafp1 • AFP is cafp1 • AFP captures information flow pattern without redundancy @!r2 p(f(X)) :- q(X). @!r4 q(g(X)) :- p(X).

  17. Answer Flow Analysis (cont’d) • An AFP can be represented as an answer-flow graph AFG = (N,E), where • N: the set of subgoals in afp • E: {(child,parent) | (child,parent) ∈ afp} • Loops in AFG represent cycles that cause non-termination

  18. Answer Flow Analysis (cont’d) • As before, we assume all predicates are tabled + subgoal abstraction • Theorem (Soundness of the answer flow analysis) If the complete trace of a query has an AFP then the query does not terminate. • Theorem (Completenessof the answer flow analysis) If the query evaluation does not terminate, then: • There is an AFP in its complete trace, • AFG = (N, E) contains at least one loop, • Every sub ∈ N appears in at least one loop, and • Each edge (sub1,sub2)∈E, where sub1=pred(..., ruleid), tells us that sub2 calls sub1 from the body of a rule whose id is ruleid.

  19. More on UCG and AFG • Theorem (Relationship between UCG and AFG) Consider the UCG and AFG for a non-terminating forest log, we have: • nodes(AFG) ⊂ nodes(UCG) • edges(AFG) ⊂ reverse-edges(UCG) • loops(AFG) ⊆ loops(UCG) • Theorem (No false results for finite traces) If the evaluation of a query, Q, terminates, then both the UCG and the AFG for Q’s trace are empty.

  20. Auto-Repair of Rules • A query does not terminate if • It has infinitely many answers, or • It has a finite number of answers, but one of its subqueries has an infinite number of them • In this case: a different evaluation order may terminate the query • This case is targeted by our auto-repair heuristic • For each unfinished(child,parent,timestamp) • We know • the host rule for this call, and • the common set of the unbound arguments of parent and child – the arguments whose bindings are to be derived • Thus, to reduce the possibility that parent receives infinite number of bindings from child, one can delay issuing a child-call from its host rule until these arguments are bound

  21. Auto-Repair of Rules (cont’d) • Example @!r1 p(a). @!r5 r(X) :- r(X). @!r2 p(f(X)) :- q(X). @!r6 r(X) :- p(X), s(X). @!r3 q(b). @!r7 s(f(b)). @!r4 q(g(X)) :- p(X). ?- r(X). Its unfinished calls are: unfinished(r(_h9900,_h9908), root, 0) unfinished(r(_h9870,r5), r(_h9870,_h9889), 8) unfinished(r(_h9840,r5), r(_h9840,r5), 11) unfinished(p(_h9810,r6), r(_h9810,r5), 12) unfinished(q(_h9780,r2), p(_h9780,r6), 16) unfinished(p(_h9750,r4), q(_h9750,r2), 20) unfinished(q(_h9720,r2), p(_h9720,r4), 24) • Applying auto-repair @!r1 p(a). @!r5 r(X) :- wish(ground(X))^r(X). @!r2 p(f(X)) :- wish(ground(X))^q(X). @!r6 r(X) :- wish(ground(X))^p(X), s(X). @!r3 q(b). @!r7 s(f(b)). @!r4 q(g(X)) :- wish(ground(X))^p(X). ?- wish(ground(X))^r(X). • Then the query will terminate with X = f(b)

  22. Tabled Engines without Subgoal Abstraction • Additional cause of non-termination: infinite number of subgoals • Steps • Compute the sequence of unfinished subgoals • Compute simplified subgoal sequence (SSS) out of unfinished subgoal sequence • Each unfinished subgoal, predicate(…, ruleid), is simplified to predicate(ruleid) • Find the SSS pattern, as in the case of answer flow pattern • SSS pattern contains the predicates and their rule ids that recursively call one another to form increasingly deep subgoals

  23. Tabled Engines without Subgoal Abstraction (cont’d) • Example @!r1 p(a). @!r4 r(X) :- r(X). @!r2 p(X) :- q(f1(X)). @!r5 r(X) :- p(X), s(X). @!r3 q(X) :- p(f2(X)). @!r6 s(a). ?- r(a). Its unfinished calls are: unfinished(r(a,_h46), root, 0). unfinished(r(a,r4), r(a,_h27), 8). unfinished(r(a,r4), r(a,r4), 11). unfinished(p(a,r5), r(a,r4), 12). unfinished(q(f1(a),r2), p(a,r5), 16). unfinished(p(f2(f1(a)),r3), q(f1(a),r2), 19). unfinished(q(f1(f2(f1(a))),r2), p(f2(f1(a)),r3), 22). unfinished(p(f2(f1(f2(f1(a)))),r3), q(f1(f2(f1(a))),r2), 25). unfinished(q(f1(f2(f1(f2(f1(a))))),r2), p(f2(f1(f2(f1(a)))),r3), 28). unfinished(p(f2(f1(f2(f1(f2(f1(a)))))),r3), q(f1(f2(f1(f2(f1(a))))),r2), 31). unfinished(q(f1(f2(f1(f2(f1(f2(f1(a)))))),r2), p(f2(f1(f2(f1(f2(f1(a)))))),r3), 34). …… • SSS = [root, r(_), r(r4), r(r4), p(r5), q(r2), p(r3), q(r2), p(r3), q(r2), p(r3), q(r2)] • SSS pattern = [p(r3), q(r2)] – says that it is predicate p of r3 and predicate q of r2 that recursively call each other, thus forming increasingly deep nested subgoals

  24. Status • Unfinished-call/answer flow implemented in SILK and Flora-2 • SILK has a GUI, Flora-2’s underway • Rule Ids are crucial for practicality • Auto-repair: not implemented yet

  25. Experiments • System • Dual core 2.4GHz Lenovo X200 with 3GB RAM • Ubuntu 11.04 with Linux kernel 2.6.38 • Small programs • They took a tiny fraction of a second to analyze • Correctness of analysesis manually verified • Large programs: one biology ontology from SILK • KB size: • Flora-2 program with 4,774 rules and 919 facts • Compiled into XSB’s 5,500+ rules and 1,000+ facts • Logs produced until evaluation consumed all memory • Size: ~2GB • Number of records: ~14M • Took 170 seconds

  26. Conclusions • Terminyzer – a tool for analyzing non-termination • Future work: • Implement auto-repair • better auto-repair algorithms • Comparison with others: • All other work deals with underpowered logic engines that are so last Century (Prolog) • Or with trying to find sufficient conditions for termination (different focus)

  27. Thank you!

  28. Forest Logging – Example :- table path/2. edge(1,2). edge(1,3). edge(2,1). path(X,Y) :- edge(X,Y). path(X,Y) :- edge(X,Z), path(Z,Y). ?- path(1,Y).

More Related