Environments and Sensing

1. Environments and Sensing Koen Hindriks Delft University of Technology, The Netherlands Multi-Agent Systems Course

2. Outline for Today Previous lecture: core Goal language: • Knowledge, beliefs, goals, actions, rules Today: • Agent Program Structure • Macros • Environments, sensing, and acting • BW4T & Agent design guidelines

3. Agent program structure

4. An Agent is a Set of Modules init module{ knowledge{ … } beliefs{ %%% INITIAL BELIEFS ONLY IN INIT MODULE %%% } goals{ … } program{ %%% PROCESS “SEND ONCE” PERCEPTS HERE %%% } actionspec{ %%% SPECIFY ENVIRONMENT ACTIONS HERE %%% } } main module{ % OPTIONAL knowledge section % NO beliefs section HERE! % OPTIONAL goal section (not advised in ‘main’) program{ %%% ENVIRONMENT ACTION SELECTION HERE %%% } } event module{ program{ %%% PROCESS PERCEPTS HERE %%% %%% PROCESS MESSAGES HERE %%% %%% PERFORM GOAL MANAGEMENT HERE %%% } } Built-in modules: • init module: • Define global knowledge • Define initial beliefs & goals • Process “send once” percepts • Specify environment actions • main module • Action selection strategy • event module • Process percepts • Process messages • Goal management User-defined modules.

5. GoalAgent Cycle Process events (= apply event rules) event module Select action (= apply action rules) main module Perform action (= send to environment) Also called reasoning or deliberation cycle. Goal’s cycle is a classic sense-plan-act cycle. Update mental state (= apply action specs + commitment strategy)

6. macros

7. Macro Definitions instead of writing mental state conditions: you may use macros in rules: mental state condition is macro’s definition program{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). } program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table). }

8. Macro Definitions • Use to increase readability • Macro definitions must be placed inside the program section, before any of the rules • All variables in macro should occur in defining mental state condition. • Other vars in condition are just placeholders. program{ #define misplaced(X) a-goal(tower([X|T])). #define constructiveMove(X,Y) a-goal(tower([X,Y|T])), bel(tower([Y|T])). if constructiveMove(X,Y) then move(X,Y). if misplaced(X) then move(X,table). }

9. Macro Definitions • When is a block marked as obstructing? Informally explain the macro definition. • In which cases is the concept of an obstructing block useful? knowledge{ … above(X,Y) :- on(X,Y). above(X,Y) :- on(X,Z), above(Z,Y). tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } program{ #define inPosition(X) goal-a( tower([X|T]) ). #define obstructingBlock(X) a-goal( on(Y,Z) ), bel( above(X,Z); above(X,Y) ). … } EXERCISE

10. Outline • Environments and Perception • Built-in Actions and Selecting Multiple Actions • Environments and Durative Actions

11. This Lecture Percepts agent events beliefs environment actions goals Action plans

12. Demo: towerworld

13. TowerWorld Environment • The TowerWorld is: • Multi-agent: user can also change the world • Fully observable: the agent can see “everything” • Dynamic: environment can change if agent does nothing • We need sensing / percepts.

14. Sensing and Perception

15. Sensing • Agents need sensors to: • explore the environment when they have incomplete information (e.g. Wumpus World) • keep track of changes in the environment that are not caused by itself • Goal agents sense the environment through a perceptual interface defined between the agent and the environment. • Environment generates perceptions

16. Sensing in the Tower World Percepts are received by an agent in it’s percept base. The reserved keyword percept is wrapped around the percept content, e.g. percept(block(a)). Not automatically inserted into beliefs!

17. Inspecting the Percept Base Percept base is inspected using the bel operator, e.g. bel( percept(on(X,Y)) ). Often useful to check for difference with current beliefs: • Agent does not believe something it perceives: bel( percept(on(X,Y)), not(on(X,Y)) ) • Agent believes something it does not perceive: bel( on(X,Y), not(percept(on(X,Y))) ) Why not: bel( not(percept(on(X,Y))), on(X,Y) )? Evaluate with only: percept(on(a,b))and on(a,c) EXERCISE

18. Processing Percepts • The percept base is refreshed, i.e. emptied and then filled again, every cycle of the agent. • Agent has to decide what to do when it perceives something, i.e. receives a percept. • Use percepts to update agent’s mental state: • Ignore the percept • Update the beliefs of the agent • Adopt a new goal

19. Built-in Actions Goal provides a number of built-in actions: • insert(<conjunction>) example: insert(on(X,Y), not(on(X,Z))) meaning: addon(X,Y) and deleteon(X,Z) • delete(<conjunction>) example: delete(on(X,Y), not(on(X,Z))) meaning: delete on(X,Y) and addon(X,Z) • adopt(<conjunction of positive literals>) example: adopt(on(a,b), on(b,c)) meaning: add new goal (if not already implied by a goal nor believed) • drop(<conjunction>) example: drop(on(b,a), not(on(c,table))) meaning: remove all goals that imply <conjunction> from the goal base Note All actions need to be closed when they are executed.

20. Built-in Action Insert Example 1: Example 2: • Belief base affected, sometimes also goal base affected. • Example 2 preferred: Use insertonly for inserting. beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,b). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } insert(on(a,b), not(on(a,table))) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). on(d,table). } goals{ on(a,b), on(b,table). } insert(on(d,table))

21. Built-in Action Delete Example 1: Example 2: Only belief base affected, if used as in Example 2 (preferred). In that case, goal base always remains unchanged beliefs{ on(a,b). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } delete(on(a,b), not(on(a,table))) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } delete(on(a,table))

22. Multiple Actions in Rule • Using the + operator multiple actions can be combined and performed in a single rule. • The actions are executed in same reasoning cycle • Combined actions are executed in order (left-to-right) • Example: rule to update moving state of BW4T robot. Why not: insert( state(NewState) ) + delete( state(OldState) )? If OldState = NewStateinformation about state would be deleted! if <mental_state_condition> then <action1> + <action2> + … . program{ if bel(state(OldState), percept(state(NewState)) ) thendelete( state(OldState) ) + insert( state(NewState) ). ….. } EXERCISE

23. Built-in Action Adopt Example 1: Example 2(a) and (b): • Only goal base affected, belief base remains unchanged. beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). on(c,a). } adopt(on(c,a)) adopt(on(a,table)) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } does not execute adopt(on(a,b)) does not execute

24. Built-in Action Drop Example 1: Example 2: • Only goal base affected, belief base remains unchanged. • Like insert and delete actions always executes. beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ } drop(on(a,b), on(b,table))) beliefs{ on(a,table). on(b,table). on(c,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,table). on(b,table). on(c,table). } goals{ } drop(on(a,b))

25. Drop Action EXERCISE • drop(on(b,a), not(on(c,table))) • Is goal in goal base dropped? • Check: does goal imply on(b,a), not(on(c,table)) ? • A: Yes, so goal is removed by drop action. knowledge{ block(X) :- on(X, Y). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]):- on(X,Y),tower([Y|T]). } goals{ on(a,table), on(b,a), on(c,b), on(d,table), on(e,table), on(f,e), on(g,f), on(h,g), on(i,h). }

26. Changing Agent’s Mind: Goals Changing One’s Mind Pattern: • If a “better” goal G2 can be set, then replace a current goal G1 with that new goal G2. • Example in Tower World Agent: program{ … if goal( G1 ), <INSERT_CONDITION_BETTER( G2 )> then drop( G1 ) + adopt( G2 ). … } program{ … if goal( holding(X) ), bel( not(clear(X) ), constructiveMove(Z,Y) then drop( holding(X) ) + adopt( holding(Z) ). … }

27. Updating Agent’s Mental State One way to update beliefs with percepts: • First, delete everything agent believes. Example: remove all block and on facts. • Second, insert new information about current state provided from percepts into belief base. Example: insert block and on facts for every percept(block(…)) and percept(on(…)). Assumes that environment is fully observable with respect to block and on facts. Downside: not very efficient…

28. Percept Update Pattern A typical pattern for updating is: Rule 1 If the agent • perceives block X is on top of block Y, and • does not believe that X is on top of Y Then inserton(X,Y)into the belief base. Rule 2 If the agent • believes that X is on top of Y, and • does not perceive block X is on top of block Y Then remove on(X,Y)from the belief base. Assumes full observability with respect to on facts.

29. Percepts and Event Module • Percepts are processed in Goal by means of event rules, i.e. rules in the event module. • Event module is executed every time that agent receives new percepts. event module{ program{ <… rules …> } }

30. Implementing Pattern Rule 1 Rule 1 If the agent • perceives block X is on top of block Y, and • does not believe that X is on top of Y Then inserton(X,Y)into the belief base. INCORRECT! event module { program{ % assumes full observability. if bel( percept(on(X,Y)), not(on(X,Y)) ) then insert(on(X,Y)). … } }

31. Implementing Pattern Rule 1 Rule 1 If the agent perceives block X is on top of block Y, and does not believe that X is on top of Y, then inserton(X,Y)into the belief base. We want to apply this rule for all percept instances that match it! insert(on(a,table)) insert(on(b,table)) insert(on(c,table)) insert(on(d,table)) … Content Percept Base percept(on(a,table)) percept(on(b,table)) percept(on(c,table)) percept(on(d,table)) … Assuming empty belief base event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). … } }

32. Implementing Pattern Rule 2 Rule 2 If the agent • believes that X is on top of Y, and • does not perceive block X is on top of block Y Then remove on(X,Y)from the belief base. • We want that all rules are applied! By default the event module applies ALL rules in linear order. • Note that none of these rules fire if nothing changed. event module { program{ % assumes full observability. forall bel(percept(on(X,Y)), not(on(X,Y))) do insert(on(X,Y)). forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). } }

33. Event Module for Tower World Rules for every type of percept, e.g., for Tower World: • blocks present; • whether gripper is holding a block or not; • a block being on top of another or on the table. event module { program{ % assumes full observability. forall bel( block(X), not(percept(block(X))) ) do delete( block(X) ). forall bel( percept(block(X)), not(block(X)) ) do insert( block(X) ). forall bel( holding(X), not(percept(holding(X))) ) do delete( holding(X) ). forall bel( percept(holding(X)), not(holding(X)) ) do insert( holding(X) ). forall bel( on(X,Y), not(percept(on(X,Y))) ) do delete( on(X,Y) ). forall bel( percept(on(X,Y)), not(on(X,Y)) ) do insert( on(X,Y) ). } }

34. Initially… Agent Knows Nothing • In most environments an agent initially has no information about the state of the environment, e.g. Tower World, Wumpus World, … • Represented by an empty belief base: • There is no need to include a belief base in this case in a Goal agent. • It is ok to simply have no belief base section. beliefs{ }

35. Processing Percepts Types of percepts: • “send always” • “send once”, e.g., place(<PlaceID>) • “send on change”, e.g., at(<PlaceID>) • “send on change with negation”, e.g., in(<RoomID>) How to handle these different type of percepts? See previous slides

36. Pattern for Send Once “Send once” percepts are sent only once when the agent first connects to an environment. Use the init module to insert the beliefs you want to use in the agent’s belief base. init module { … program{ forall bel( percept(place(X) ) do insert( place(X) ). … } }

37. Send on Change Percept • “send on change” percepts are sent once when a change occurs: • Note that pattern for “send always” percepts does not work for “send on change” due to second rule in that pattern. - state(traveling) state(arrived) State 2 State 3 State 4 State 1 State 5 event module { % send always pattern program{ … forall bel(on(X,Y), not(percept(on(X,Y)))) do delete(on(X,Y)). } }

38. Pattern for Send on Change “Send on change” percepts are sent only when something in the environment has changed. Rule: remove old belief and insert the new percept. Put rule in event module. event module { … program{ forall bel( state(Old), percept(state(New) ) do delete(state(Old)) + insert(state(New)). … } } Could also have used ‘if’ here: At most one state percept received.

39. Send on Change with Negation “Send on change with negation” percepts: • Positive fact sent once when it becomes true in environment • Negative fact sent once when it becomes false in environment Rule: insert positive and remove negative facts. Put rules in event module. event module { … program{ forall bel(percept(in(RoomID)) do insert(in(RoomID)). forall bel(percept(not(in(RoomID))) do delete(in(RoomID)). ... } }

40. Combining Information In BW4T it is most important to remember in which place a colored block can be found. Idea: combine at location with ‘color’ percept using a new ‘block’ predicate. NB: make sure by earlier rule that at belief is correct! event module { … program{ … forall bel( percept(color(BlockID, ColorID)), at(PlaceID) ) do insert( block(BlockID, ColorID, PlaceID) ). forall bel( at(PlaceID), block(BlockID, ColorID, PlaceID), percept(not(color(BlockID, ColorID))) ) do delete( block(BlockID, ColorID, PlaceID) ). … } }

41. Summarizing • Two types of rules: • if <cond> then <action>. is applied at most once (if multiple instances chooses randomly) • forall <cond> do <action>. is applied once for each instantiation of parameters that satisfy condition. • Main module by default: • checks rules in linear order • applies first applicable rule (also checks action precondition!) • Event module by default: • Checks rules in linear order • Applies all applicable rules (rules may enable/disable each other!) • Program section modifiers: [order=random], [order=linear], [order=linearall], [order=randomall] • Built-in actions: insert, delete, adopt, drop.

42. Actions in environments

43. Instantaneous versus Durative • Instantaneous actions Actions in the Blocks World environment are instantaneous, i.e. they do not take time. Wumpus World actions are of this type as well. • Durative actions Actions in the Tower World environment take time. When a Goal agent sends an action to such an environment, the action will not be completed immediately.

44. Durative Actions Deciding on a next action typically gets more complicated when actions take time! Agents may decide to perform an action before finishing another. Three cases: • Sending an action B while another action A is still ongoing overrides and terminates previous action A… Example: actions in Tower World, goto action in UT2004. • Action B that is sent while another action A is still ongoing is performed in parallel… both actions are performed. Example: shooting while running in UT2004. • Action B that is sent while another action A is still ongoing is simply ignored… nothing new happens. Less frequent.

45. Durative Actions and Sensing • While durative actions are performed an agent may receive percepts. • Useful to monitor progress of action. • Tower World Example: Gripper may no longer be holding block because user removed it. • BW4T Example: Other robot is blocking entrance to room.

46. Specifying Durative Actions • Tower World has two actions: pickup and putdown block both take time. • How should we specify these actions?  delayed effect problem. Solution: • do not specify postcondition, • make sure action effects are handled by event rules.

47. Specifying Durative Actions Tower World action specification: Postcondition may also be “empty”: post { } Better practice is to indicate that you have not forgotten to specify it by using post { true }. actionspec{ pickup(X) { pre{ clear(X), not(holding(Y)) } post{ true } } putdown(X,Y) { pre { holding(X), clear(Y) } post { true } } nil { pre { true } post { true } } }

48. Multiple Environment Actions • Warning: It may not be useful to combine environment actions. • Environment actions may cancel each other out • What happens with the following rule? • Nothing… if gripper is already at “home position” program{ … if goal( holding(X) ) then pickup(X) + nil. … }

49. Multiple Environment Actions • Warning: It may not be useful to combine environment actions. • Environment actions may cancel each other out • What happens with the following rule? • If robot is at “PlaceID” then maybe no pickup! program{ … if bel( block(BlockID, ColorId, PlaceID) ), goal( holding(ColorId) ) then goTo(PlaceID) + pickUp(BlockID) + goTo(‘DropZone’) . … }

50. BW4T ASSIGNMENT