Plan of the tutorial

1. Plan of the tutorial • Introduction • Normative systems • The agent perspective: normative multiagent systems • The construction of social reality website: http://normas.di.unito.it/iat04

2. Introduction

3. Social norms In the Multi­Agent Systems field, social norms are perceived to help improve coordination and cooperation (Shoham & Tenneholz 1992; Jennings and Mandami 1992; Conte & Catselfranchi 1995; Jennings 1994; Walker & Wooldridge 1995). Agents cannot be assumed to be benevolent

4. Why norms? (a) How to avoid interferences and collisions among agents autonomously acting in a common space? (b) How to ensure that negotiations and transactions fulfil the norm of reciprocity? (c) How to obtain a robust performance in teamworks? (d) How to prevent agents from dropping their commitments, or how to prevent agents from disrupting the common activity ?

5. Shoham & Tenneholz 1992 In multiagent systems be they human societies or distributed computing systems different agents, people or processes, aim to achieve different goals and yet these agents must interact either directly by sharing information and services or indirectly by sharing system resources. In such distributed systems it is crucial that the agents agree on certain rules in order to decrease conflicts among them and promote cooperative behavior. Without such rules even the simplest goals might become unattainable by any of the agents or at least not efficiently attainable. Just imagine driving in the absence of traffic rules. These rules strike a balance between allowing agents sufficient freedom to achieve their goals and restricting them so that they do not interfere too much with one another

6. Shoham & Tenneholz 1992 They consider the possibility of limiting the agents to a subset of the original strategies of a given game thus inducing a subgame of the original one. They call such a restriction a social constraint if the restriction leaves only one strategy to each agent. Some social constraints are consistent with the principle of individual rationality in the sense that it is rational for agents to accept those assuming all others do as well.

7. Hard or soft constraints? The distinction between hard and soft constraints corresponds to the distinction between preventative and detective control systems. In the former a system is built such that violations are impossible (you cannot enter metro station without a ticket) or that violations can be detected (you can enter train without a ticket but you may be checked and sanctioned).

8. Autonomous agents • Agents: systems oriented to achieve states in the world. • Goal: an explicit representation of a world state which the agent wants to be realised; agents with goals and beliefs are cognitive agents. • Belief: a representation of the world that the agent holds true. • Norm: an obligation on a set of agents to accomplish/abstain from a given action, • external: no mental representation • internal, • Institution: a supra-individual system deliberately designed or spontaneously evolved to regulate agents’ behaviour. • Autonomy: an agent is autonomous wrt • its physical environment or • other agents in the same environment -> social autonomy. • Goal-autonomy • Norm-autonomy

9. Autonomy "...to pose a goal to oneself is something about which no external legislation can interfere...". An agent: "cannot undergo any obligation other than what he gives himself on his own. (...) only by this means it is possible to reconcile this obligation (even if it were an external obligation) with our will". Kant (Die Metaphysik der Sitten, 1794)

10. Normative systems anddeontic logic

11. Normative systems • “Sets of agents whose interactions are norm-governed; the norms prescribe how the agents ideally should and should not behave. [...] Importantly, the norms allow for the possibility that actual behavior may at times deviate from the ideal, i.e., that violations of obligations, or of agents’ rights, may occur.” (Jones & Carmo 2001)

12. Deontic logic • von Wright, 1951: formal study of ought • Deontic modalities besides alethic ones • “it is obligatory to see to it that x” inspired to “it is necessary that x” • “it is permitted to see to it that x” inspired to “it is possible that x”

13. Modal operator • Op = it is obligatory that p • Pp = Op it is permitted that p • Fp = Op it is forbidden that p • Minimal system D: • O(p  q)  (Op  Oq) • O(p)  Op (I.e. Pp, obligatory implies permitted) • if |- α then |- Oα (but not Op  p like for knowledge: ideal is not real necessarily)

14. Conditionals and paradoxes • Obligations are inherently conditional:“when you… you have to…” • Different possibilities to define O(y|x) • x  O(y) • O(x  y) • NEC(x  O(y)) • They all have counterintuitive results: paradoxes of deontic logice.g. contrary to duty O(kill) but O(gently|kill)

15. Anderson’s reduction I • Reduction of deontic logic to alethic logic: “the intimate connection between obligations and sanctions in normative systems suggests that we might profitably begin by considering some penalty or sanction S, and define obligations as: p is obligatory if its falsity entails the sanction S”.

16. Anderson’s reduction II • Formalization • O(p) = NEC(p  S) • S • Problem: not all violations are sanctioned • Reply of Anderson “S just means something bad or violation”

17. Dynamic logic • Meyer, 1988: Deontic logic viewed as a variant of dynamic logic • a is obligatory if the effect of not doing action a is that there is a violation:O(a) = [¬a] V

18. The agent perspective: normative multiagent systems

19. Social order I Castelfranchi 2000: Social orders are patterns of interactions among interfering agents that allow the satisfaction of the interests of agents, such as values or shared goals that are beneficial for most or all of the agents Agents delegate to the normative system their own shared goals which become the content of the obligations regulating the system

20. Social order II For example, if agents delegate the goal to avoid accidents to the normative system, then the system may adopt the subgoal to drive on the right side of the street. This subgoal is the content of the obligation to regulate traffic. Agents adopt this goal since they contribute to the delegated goal, and they know other agents will adopt it too

21. Social control Castelfranchi 2000: Social control “An incessant local (micro) activity of its units aimed at restoring the regularities prescribed by norms”. Agents attribute to the normative system the ability to autonomously enforce the conformity of the agents to the norms

22. Violating norms Probably, when one thinks about multiagent systems, one assumes that the agents stick to the obligation posed by the system. However, this assumption is not always realistic, so we must consider what happens with agents that must be motivated to respect an obligation. See heterogeneous multi-institutional agents, like the Grid.

23. Why violating norms Nobody can avoid that norms - and in particular their instances - might be incoherent. There might be conflicts, and the agents should be able to manage these conflicts. Norms also cannot predict and successfully frame all possible circumstances. There might be some important event or fact to be handled, where no norm applies or some norm applies with bad results.

24. Instrumental norms • Law scholars like Hart distinguish: • primary norms: prescription of behavior • instrumental norms: help the achievement of primary norms. Directed towards the juridical system: sanctions, procedures for trials • … (deontic logic focussed on primary norms)

25. What do we learn from this? • Mental attitudes like goals not only at the individual level: delegation of goals • Normative systems: not only specification of ideal behavior of the system, but also active role • Normative system has goals and does actions. Is it an agent?

26. The agent metaphor • G.Lakoff: Role of metaphor in cognition to conceptualize reality which is not bodily grounded. • An ontology of social reality should disclose the metaphorical mapping we use to understand social reality • Can the agent metaphor be used for understanding social reality?

27. Intentional stance • Dennet: attitudes like belief and desire are folk psychology concepts thatcan be fruitfully used in explanations of rational human behavior.For an explanation of behavior it does not matter whether oneactually possesses these mental attitudes: we describe thebehavior of an affectionate cat or an unwilling screw in terms ofmental attitudes. Dennet calls treating a person or artifact as arational agent the ‘intentional stance’.

28. The importance of us • “The possibility of ascribing goals, beliefs, and actions tocollectives relies on the idea that collectives can be taken toresemble persons. […] both factual and normative beliefs can be ascribed(somewhatmetaphorically) to groups, both formal and informal, structuredand unstructured.” Tuomela, 1995

29. Norms as mental attitudes(Boella and van der Torre) • If a normative system is described as an agent with mental attitudes,thus norms are defined in terms of the conditionalmental attitudes of the normative agent • obligations are goals (“ideal behavior”) • what about beliefs?

30. Input/Output Logics(Makinson & van der Torre) • Let R  Rul: a,..,d→x or (a,…d,x) • Outi(R) is closure under set of rules • Out1:SI Out2:SI,OR • Out3:SI,CT Out4:SI,OR,CT a→x a,b→x a,b→x a→b a,b→x CT SI OR a,b→x a→x a→x • Outi+: Outi and ID ID a→a

31. Multiagent system MAS=<A,X,G,E,> • A: set of agents • X: propositional variables • G: goal rules a,..,d→x • E: effect rules a,..,d→x • : priority relation on goal rules Xa: actions of agent a, Ga: goals of a

32. Normative MAS NMAS=<A,X,G,E,,N,V,n> • n  A the normative agent • N: a set of norms • V: norm description N x A → Xeg V(n,a) Anderson’s reduction: Obligation Oa(x,s|C) if • C, x → V(n,a)  E • V(n,a) → s E

33. Alternative approach • Violation is not an effect of the behavior, but an action of the normative system • Analogously, the sanction is an action of the normative system (with a cost) • Recognizing violations and sanctioning violations are goals of the normative system

34. Obligations Oa,NS(x,s|Y) • Y→x is goal of NS • Y,x → V(n,a) is goal of NS • Y,V(n,a) → s is goal of NS • Y → s is goal of agent a Two actions: V(n,a) = violation by agent a of norm n s is a sanction Hart: Instrumental norms precondition

35. Michael Luck, Fabiola López y López • Societies and Autonomous Agents. • How can autonomous agents be integrated into societies regulated by norms? • What does an agent need to deal with norms? • What does an agent evaluate before dismissing a norm? • How are the goals of an agent affected by social regulations?

36. Michael Luck, Fabiola López y López • A formal structure of norms that includes the different elements that must be taken into account when reasoning about norms • A formal basic representation of norm-based systems • An analysis and formalisations of the kinds of norms that norm-based systems have • An analysis of the dynamics of norms • The set of normative relationships that might emerge by adopting, complying and dismissing norms

37. Spread Modification Adoption Abolition Activation Compliance Violation Dismissal Reward Sanction Non-sanction Norms dynamics Issue

38. hindered by normative gs goals normative goals Norms compliance current goals benefited from rewards hindered by punishments

39. Z specification

40. Z specification

41. Emergence of norms Off-line design: In this approach, social laws are designed off-line, and hard-wired into agents (Shoham & Tennenholtz 1992b; Goldman & Rosenschein 1993; Conte & Castelfranchi 1993). Emergence from within the system: (Shoham & Tennenholtz 1992a; Kittock 1993), a convention can ‘emerge’ from within a group of agents. The first approach will often be simpler to implement, with a greater degree of control over system functionality. But not all the characteristics of a system are known at design time; not suited for open systems.

42. Conte, Castelfranchi, Dignum, 1998 • Social science: norms are emergent properties of utility driven behavior. (Binmore 1994) • They survive if associated with monitoring and sanctioning (Axelrod 1987, Boyd 2003) • Social science does not explain the decision process of autonomous agents

43. Norm acceptance • Norms would not be respected if there were the sanction only: 90% of crimes are not punished • Norms are respected since they are accepted • They derive from goals delegated to the normative system

44. Autonomous norm acceptance An agent is norm­autonomous if it can: (a) recognise or not a norm as a norm (normative belief formation); (b) argue whether a given norm concerns or not its case; decide to accept the norm or not; (c) decide to comply or not with it (obey or violate); (d) take the initiative of re­issuing (prescribing) the norm, monitoring, evaluating and sanctioning the others' behaviour relatively to the norm.

45. Goal Acceptance • Goal-acceptance= a special case of goal-generation: social goal-filter. • IF x wants p, and • x believes that IF y obtains q • THEN x obtains p • THEN x wants that y obtain q. • Autonomous agents accept a new goal iff they believe that it is a means for an old one. • The value of a current goal p increases if agents (are led to) believe that p is • Instrumental to one more important (meta-)goal q, or more (meta-)goals Q (instrumentality beliefs. These include beliefs about achievement costs). • Probability of instrumental connection is higher than expected (probability beliefs, whose credibility increases as a function of credibility of sources. These include a different evaluation of feasibility). • Endangered. Maintenance goals are more compelling than achievement ones (emergency beliefs).

46. Norm acknowledgement Input = a candidate norm (external norm). An obligation in the form OyX( q), q = the norm, y = authority that issues the norm and X = the set of the norm subjects. Output = possibly a normative belief. Several tests: evaluation of the c- norm: is it based on a recognised N? evaluation of the source: Is agi entitled to issue N? This entails: is q within the domain of y 's competence? is the current context the proper context of q? is X within the scope of y 's competence? evaluation of the motives: is q issued for agi 's personal motives? The evaluation process is formalised as follows: BELx(OzU( r)) & BELx(OzU( r)  OyX( q)) (10) (OyX( q) & BELx(auth(y,X,q,C)) & BELx(mot(y,OK)))  BELx(OyX( q)) (11) Both lead to BELx(OyX( q)) The relation “auth”: y has authority to issue q on X in C. The relation “mot”: y's motives are correct.

47. Acceptance (From Conte et al., 1998) Is N-belief sufficient? No! Belief about instrumentality. Normative corollary of social autonomy: x will form a N-goal q iff it believes that q is instrumental to a further goal: BELx(OyX( q)& INSTR(OBTX(q),p) &GOALx(p|r))  N-GOALx(OBTX(q)|GOALx(p|r) & r) Important differences from the g-generation rule: the existence of a N-belief. But norms can be autonomously created: BELx(O(OyX( q)) & INSTR(OBTX(q),p)& GOALx(p|r))  N-GOALx(OBTX(q)|GOALx(p|r) & r) the form of the instrumental belief. But x may have internalised the norm: BELx(OyX( q) & INSTR(q,p) & GOALx(p|r))  C-GOALx(q|GOALx(p|r) & r)No N-conformity. We need: BELx(BELy(OzX( q)))  BELx(OzX( q)) BELx(N-GOALy(OBTX(q)| r)  INSTR(OBTX(q),be_like(x,y))) plus GOALx(be_like(x,y)|true)

48. So far... • Agents undergo social influence, that is they are often implicitly or explicitly requested to accept new goals. • Institutional influence is a special case of social influence. • In both cases, autonomous agents accept new goals (including normative ones) only as means to achieve old ones. • Questions • But what are the specific motives for accepting influence and forming new goals? • What is their respective efficacy? Which type of influence is more effective?

49. Norm (old) Goal (execute action) Motives for Acceptance Goal (old) Bel (p of connection) Goal (execute action) • Trust (probability/emergency belief) • Acknowledgement • Social Responsibility • Don’t harm • Material (e.g., passive smoking) • Symbolic harm (break institutional authority) • Don’t give a bad example Emotions Norm (acceptance) Bel (instrumentality) Bel (emergency)

50. Goal (avoid penalty Bel (instrumentality) Norm (acceptance) Goal (old) Side-Goal Bel (importance) Bel (importance) Goal (execute action) Meta-Goal Goal (old) Goal (execute action) Motives for Acceptance (cont’) Incentives • Negative • penalty • costs of action • obstacles • Positive • side-goals • meta-goals