1 / 42

Relational Preference Rules for Control

30.6.2009. Relational Preference Rules for Control. Participants: Raphael Aroch Tomer Shiri Advisor: Prof. Ronen Brafman. http://project.shiri.info. Overview. Background Example System Structure Rule generation Algorithms: Naïve Algorithm Local Search Branch & Bound

monte
Download Presentation

Relational Preference Rules for Control

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 30.6.2009 Relational Preference Rules for Control Participants: Raphael Aroch Tomer Shiri Advisor: Prof. Ronen Brafman http://project.shiri.info

  2. Overview • Background • Example • System Structure • Rule generation • Algorithms: • Naïve Algorithm • Local Search • Branch & Bound • Reduction to Relational Probabilistic Models • Experiments • Summery

  3. Background Introduction Relational Preference Rules are usedin order to solve real – world decision making problems. This can be used for the fire department of large city, a control room for handling large scale emergencies such as earthquakes, flooding or terrorist attacks. 10/11/2014 3

  4. Background Goals Our project we simulate real world conditions in order to empirically test the efficiency and scalability of various search algorithms. In our project we will simulate a fire department control room. 10/11/2014 4

  5. Background Structure The system contains a set of objects and a set of rules which create a real – world state. Every Object has some attributes. Camera Location Rank • Fireman‘s camera can be controlled – turned On or Off, thus controllable. • A fireman‘srank or location is fixed (in a given time), so its uncontrollable.

  6. Background Structure • Preference Rule Is of the following form: • rule-body → rule-head : < (v1,w1),…,(vn,wn) > • Rule body contains conditions in CNF. • Fireman(A).location = 10 ^ Fireman(A).location = Fireman(B).location • Rule head contains one controllable attribute and a series of pairs of values • and weights. • Fireman(A).camera : <(On, 4), (Off, 0)> 10/11/2014 6

  7. Finding an Optimal Assignment What is an optimal assignment? • An optimal assignment is a Vector of assignments to controllable attributes, • such that the sum of weights of all true evaluated rules is maximized. 10/11/2014 7

  8. Finding an Optimal Assignment Decision making example Lets evaluate the following scenario: Two firemen are both at the location of a fire. We have the following ground rules: Fireman(A) ^ Fire(B) ^ A.location = B.location → A.Camera : <On, 4> Fireman(A) ^ Fireman(B)^ Fire(F)^ A != B ^ A.location = B.location^ A.location = F.location^A.Camera = On→ B.Camera : <On , -4> And the following objects: Alice Bob Fire 10/11/2014 8

  9. Finding an Optimal Assignment Decision making example – Rule Grounding By assigning A,B and F, We get the following ground instances: • Alice ^ Fire ^ Alice.location = Fire.location → Alice.Camera : <On, 4> • Bob ^ Fire ^ Bob.location = Fire.location → Bob.Camera : <On, 4> • Alice ^ Alice^ Fire ^ Alice != Alice ^ Alice.location = Alice.location^ Alice.location = Fire.location^Alice.Camera = On→ Alice.Camera : <On , -4> • Alice ^ Bob^ Fire ^ Alice != Bob^ Alice.location = Bob.location^ Alice.location = Fire.location^Alice.Camera = On→ Bob.Camera : <On , -4> • Bob^ Alice^ Fire ^ Bob!= Alice^ Bob.location = Alice.location^ Bob.location = Fire.location^Bob.Camera = On→ Alice.Camera : <On , -4> • Bob^ Bob^ Fire ^ Bob!= Bob^ Bob.location = Bob.location^ Bob.location = Fire.location^Bob.Camera = On→ Bob.Camera : <On , -4> 10/11/2014 9

  10. Finding an Optimal Assignment Decision making example – Rule Filtering We assign all the uncontrollable attributes, evaluate the rules and filter out all the rules that evaluated as false. Alice and Bob are both at the location of the fire. Our filtered rule system will be: T → Alice.Camera : <On, 4> T → Bob.Camera : <On, 4> Alice.Camera = On→ Bob.Camera : <On , -4> Bob.Camera = On→ Alice.Camera : <On , -4> 10/11/2014 10

  11. Finding an Optimal Assignment Decision making example – Optimal Assignment Now, we will consider all the possible assignments to the controllable attributes. In this case, Alice’s and Bob’s cameras. The optimal assignment is an assignment that will return the highest value for a given rule set. In this case, assigning both cameras the same value will score 0, While assigning the cameras with different values will return 4. T → Alice.Camera : <On, 4> T → Bob.Camera : <On, 4> Alice.Camera = On→ Bob.Camera : <On , -4> Bob.Camera = On→ Alice.Camera : <On , -4> 10/11/2014 11

  12. System Structure Evaluator Evaluator Search Algorithm Config file Generator Evaluator Result 10/11/2014 12

  13. Rule Generation Parameters • Instances Amount: Number of instances of Objects in “the world”. • Classes Amount: Number of class types used – types of Objects in “the world”. • Rules Amount: Number of rules in rule system. • Conditions Amount: Number of conditions in each rule. • Max Variables Amount per Rule: Maximum number of variables in each rule. • Controllable Amount: Number of controllable attributes in rule system. • Cycles Amount: Number of size 2 cycles of rules (e.g. x → y, y → x). 10/11/2014 13

  14. Rule Generation Classes Generator Representing a kind of “Database” of objects: • Given Classes Amount determines which classes types to use out of the available classes. • Fireman Fireman • Firetruck Policeman • Policeman • Policecar • Given Instances Amount generates Classes Instances from the classes types determined above. • Fireman Alice, Bob • Policeman John, Susie 10/11/2014 14

  15. Rule Generation Rule Generator Generates rules: • A preparation stage is taking place on the Rules System level: • How many Variables in each rule. • How many Controllable Attributes in each rule. • Match couples of rules with Controllable Attributes into cycles. • A preparation stage is taking place on each individual rule: • How many terms in each condition – Unary \ Binary condition. • Which condition is controllable (contains controllable attribute\s). • According to the preparation stages, the rule generation begins: • Generating Assignment Classes. • Generating Rule Body. • Generating Rule Head. 10/11/2014 15

  16. Algorithms Naïve Algorithm Simple approach: go over all the search space and find the optimal assignment. 10/11/2014 16

  17. Algorithms Naïve Algorithm Simple approach: go over all the search space and find the optimal assignment. Max  -Infinity For every Possible assignment Ai, { value = eval_ruleset(Ai); if (value > max) value = max } 10/11/2014 17

  18. Algorithms Naïve Algorithm Simple approach: go over all the search space and find the optimal assignment. Max  -Infinity For every Possible assignment Ai, { value = eval_ruleset(Ai); if (value > max) value = max } Complexity: (# of controllable attributes)^(domain size) 10/11/2014 18

  19. Algorithms Naïve Algorithm Simple approach: go over all the search space and find the optimal assignment. Max  -Infinity For every Possible assignment Ai, { value = eval_ruleset(Ai); if (value > max) value = max } Complexity: (domain size)^(# of controllable attributes) Since we’re dealing with a “real world” problem, this is not an option. 10/11/2014 19

  20. Algorithms Local Search In “real world” problems, where a quick response is critical, we cannot afford scanning the whole search space. In this method we start with a random assignment and examine its neighbors. • An assignment is a vector <A1,…,An>, Where Ai is a controllable attribute. • A neighbor is a vector <A1,…,Bi,…,An>, Where Ai != Bi. <CameraA.on , CameraB.off> <CameraA.on , CameraB.on> <CameraA.off , CameraB.on> 10/11/2014 20

  21. Algorithms Local Search We have implemented Simulated annealing. This method occasionally selects a neighbor that is not the best one in order to avoid local maxima. Since local search has a stochastic nature, we cannot bound the number of iterations needed to find the optimal solution. assignmenteval_ruleset(Random Assignment) While (!done()) { neighbor get_neighbor(assignment) value  eval_ruleset(neighbor) if (value> value(assignment)) assignment  neighbor else if (random() < e^(value/time)) assignment  neighbor } 10/11/2014 21

  22. Algorithms Branch and Bound • An assignment is a vector <A1,…,An>, Where Ai is a controllable attribute. • Every controllable attribute ,Ai, has its own domain { d1,…,dn } 10/11/2014 22

  23. Algorithms Branch and Bound • An assignment is a vector <A1,…,An>, Where Ai is a controllable attribute. • Every controllable attribute ,Ai, has its own domain { d1,…,dn } 10/11/2014 23

  24. Algorithms Branch and Bound • An assignment is a vector <A1,…,An>, Where Ai is a controllable attribute. • Every controllable attribute ,Ai, has its own domain { d1,…,dn } A1d1 A1d2 A1d3 10/11/2014 24

  25. Algorithms Branch and Bound • An assignment is a vector <A1,…,An>, Where Ai is a controllable attribute. • Every controllable attribute ,Ai, has its own domain { d1,…,dn } A1d1 A1d2 A1d3 A2d1 A2d2 A2d3 A2d3 A2d1 A2d2 A2d3 A2d1 A2d2 10/11/2014 25

  26. Algorithms Branch and Bound • We are performing a DFS search. 10/11/2014 26

  27. Algorithms Branch and Bound • We are performing a DFS search. 10/11/2014 27

  28. Algorithms Branch and Bound • We are performing a DFS search. A1d1 A1d2 A1d3 10/11/2014 28

  29. Algorithms Branch and Bound • We are performing a DFS search. • Every node has an upper bound – maximum value for that partial assignment. ∞ A1d1 2 A1d2 A1d3 4 3 10/11/2014 29

  30. Algorithms Branch and Bound • We are performing a DFS search. • Every node has an upper bound – maximum value for that partial assignment. • When reaching a terminal node we can set a lower bound. ∞ A1d1 A1d1 2 A1d2 A1d3 4 4 3 4 4 3 A2d3 A2d1 A2d2 10/11/2014 30

  31. Algorithms Branch and Bound • We are performing a DFS search. • Every node has an upper bound – maximum value for that partial assignment. • When reaching a terminal node we can set a lower bound. • every note which its upper bound is lower than this threshold will be pruned. ∞ A1d1 2 A1d2 A1d3 4 3 4 4 3 A2d3 A2d1 A2d2 10/11/2014 31

  32. Algorithms Reduction to Relational Probabilistic Models – Markov Logic Theory Using the Alchemy Open Source AI Project: “Alchemy is a software package providing a series of algorithms for statistical relational learning and probabilistic logic inference, based on the Markov logic representation. Alchemy allows you to easily develop a wide range of AI applications” Transforming preference rule into a weighted formula, as used in Markov Logic Theories: Fireman(A) ^ Fire(B) ^ Fireman(A) .location = Fire(B).location → Fireman(A).Camera : <On , 4> <Off, 0> Is transformed into: Fireman(A) ^ Fire(B) ^ Fireman(A) .location = Fire(B).location → Fireman(A).Camera = On : 4 And Fireman(A) ^ Fire(B) ^ Fireman(A) .location = Fire(B).location → Fireman(A).Camera = Off : 0 10/11/2014 32

  33. Algorithms Reduction to Relational Probabilistic Models – Markov Logic Theory MAP/MPE Inference - Most Probable Explanation • Problem: Find most likely state of world given evidence • This is just the weighted MaxSAT problem. 10/11/2014 33

  34. Algorithms Reduction to Relational Probabilistic Models – Markov Logic Theory The MaxWalkSAT Algorithm fori ← 1 to max-triesdo solution = random truth assignment for j ← 1 tomax-flipsdo if ∑ weights(sat. clauses) > thresholdthen return solution c ← random unsatisfied clause with probabilityp flip a random variable in c else flip variable in c that maximizes ∑ weights(sat. clauses) return failure, best solution found 10/11/2014 34

  35. Experiments Aspects Tested • How algorithms scale as we increase number of rules. • How algorithms scale as we increase number of variables within a rule. • How algorithms scale as we increase number of objects. • How algorithms scale as we increase number of cycles. • For Local Search – results as a factor of time. 10/11/2014 35

  36. Experiments Results Local Search – Optimal Solution as a factor of time and rules amount 10 rules As the rules amount increases, the optimal solution takes longer to Be found. 50 rules 100 rules 200 rules 10/11/2014 36

  37. Experiments Results Local Search – Optimal Solution as a factor of time and rules amount 10 rules As time passes, more problems find the optimal solution. 50 rules 100 rules 200 rules 10/11/2014 37

  38. Experiments Results Local Search – Optimal Solution as a factor of time and objects amount 5-7 objects As objects amount increases, more iterations are needed in order to reach optimal solution. 12 objects 10/11/2014 38

  39. Experiments Results Branch & Bound – Algorithm Time as a factor of Rules Amount When Rules Amount is high, calculating the upper bound becomes a heavy operation. 10/11/2014 39

  40. Experiments Results Branch & Bound – Algorithm Time as a factor of Instances Amount For more than 25 Instances, algorithm time exceeds 10 sec. 10/11/2014 40

  41. Summary • There is technical bound on the amount of rules \ • objects tested. 2GB ~ 20 objects & 60 rules. • There is a thin line between getting good results in • reasonable time, to not getting any results. • B&B can reach optimal solution very fast, but will • spend much time seeking for other solutions. • For large search spaces, Simulated Annealing is • preferred over B&B. 10/11/2014 41

  42. Q & A

More Related