Maria Miceli Angelo Oddi Mario Paolucci  Roberto Pedone Giuliano Pistolesi  Paola Rizzo

National Research Council, Institute of Cognitive Science and Technology V.LE Marx 15, 00137 Roma Laura Benigni Sergio Benvenuto Cristiano Castelfranchi* Amedeo Cesta Rosaria Conte Rino Falcone Maria Miceli Angelo Oddi Mario Paolucci Roberto Pedone Giuliano Pistolesi Paola Rizzo *University of Siena, Science of Communication National institue of Statistics

Social & Institutional Influence.Why people accept policies Project on “Multi Agent Systems & Social Simulation”

The Problem "...to pose a goal to oneself is something about which no external legislation can interfere...". An agent: "cannot undergo any obligation other than what he gives himself on his own. (...) only by this means it is possible to reconcile this obligation (even if it were an external obligation) with our will". Kant (Die Metaphysik der Sitten, 1794) • Socialagents are autonomous • How to regulate autonomous entities?

Organization of the Talk • Autonomous agency • Principle • Two filters • Belief filter • Goal filter • Social Influence • Mechanisms • Power (e.g.,reliability) • Goal acceptance • Institutional influence • Acknowledgement • Reliability (what is it to trust an institution?) • Empowerment • Norm acceptance • Questions • Motives for acceptance • Respective efficacy.

Main concepts • Agents: systems oriented to achieve states in the world. • Goal: an explicit representation of a world state which the agent wants to be realised; agents with goals and beliefs are cognitive agents. • Belief: a representation of the world that the agent holds true. • Norm: an obligation on a set of agents to accomplish/abstain from a given action, • external: no mental representation • internal, Social norms are included. • Institution: a supra-individual system deliberately designed or spontaneously evolved to regulate agents’ behaviour. • Autonomy: an agent is autonomous wrt • its physical environment or • other agents in the same environment -> social autonomy. • Goal-autonomy • Norm-autonomy

Influence • Principle of autonomous agency (agjhas p as a new goal)  q ( agjcome toBEL (p q)) in words, agj has a new goal, if it believes that this is instrumental to an old one! • Cognitive influence of agion agj (agi GOAL(agj GOAL p))  (agi R-GOAL (agj BEL q p(p q)) (agi BEL (agj GOAL q))) in words, if agi wants to influence agj cognitively, agi will let agi believe that the world stateagi wants agj to want, achieves one of agj‘s goals.

Influencing Mechanisms • Generate a goal anew (spare power consumption) • Modify the value of a current goal (quit smoking)

Autonomous Agents • Two sequential filters • Belief filter • Goal filter • An integrated processing of mental representations: • Criteria for knowledge acquisition (filtering beliefs) and belief revision • Mechanisms for goal-activation • Goal acquisition (filtering goals), • Goal dynamics (the change of values and positions of goals);

Autonomous Agents Architecture

The Belief Filter • To believe is a decision-based state of the mind. Agents have control over their own beliefs: • Epistemic control • credibility : • coherence with previous beliefs; • reliability of the source; • introspection: the agent is the best expert about its own mind • non-negotiability. The Pascal law: we cannot use threats ("Argumentum adbaculum") or promises (non-natural means-end links) to make people believe something. • Pragmatic control • relevance : A belief is relevant for a given goal when • it is about that goal: • x wants p and • x believes he wants p • it is about the goal's propositional content: • x wants p • x believes that if q is true then p is true also • it is about the goal's planning links • x wants p • x believes that if p is true then z is true also • x wants z.

The Goal Filter • Agents have control over their own goals: • Self-interested goal-generation. • An agent is autonomous iff whatever goal q it comes to have, there is • at least a goal p of that agent, for which q is believed by that agent to be • a means. • But self-interestedness ≠ utility function: autonomous agents are not • necessarily aimed at maximizing their utility, they are aimed at achieving • their goals. • Goal-generation ≠ goal-execution: an agent may abandon its goals. • Belief-driven goal-processing: any modification of an autonomous agent's • goals can only be allowed by a modification of its beliefs. • Social consequences...

Goal-Generation • A believed antecedent of a wanted consequence is wanted: • IF x wants p, and • x believes that if q is true p will eventually be true, • THEN x wants q. • ( goal≠ executed goal…) • Beliefs provide reasons for goals...

Goal-Dynamics • … and for changing goals’ values. • If a belief is retreated, the goal is removed from its current step. • It can be interrupted and put to wait, or be completely abandoned, etc. • Sleeping (ex. survival: emergency belief -> no move) • Active (ex. now, I want to drink some water: activating perception • Waiting (I give up drinking water now, I’ll do it soon after my talk...: compatibility belief ) • Pursued (... and choose to continue my talk: preference belief) • Planned (ex. How do I keep their attention? Perhaps, I should speed up a little bit: know-how belief); • Satisfied (I am already speeking at a reasonable speed: satisfaction belief) • Dropped (perhaps I could tell a nice joke? Impossibility belief • Executed (I promise I will soon get to the end of the talk: Cando belief).

Goal Acceptance • Goal-acceptance= a special case of goal-generation: social goal-filter. • IF x wants p, and • x believes that IF y obtains q • THEN x obtains p • THEN x wants that y obtain q. • Autonomous agents accept a new goal iff they believe that it is a means for an old one. • The value of a current goal p increases if agents (are led to) believe that p is • Instrumental to one more important (meta-)goal q, or more (meta-)goals Q (instrumentality beliefs. These include beliefs about achievement costs). • Probability of instrumental connection is higher than expected (probability beliefs, whose credibility increases as a function of credibility of sources. These include a different evaluation of feasibility). • Endangered. Maintenance goals are more compelling than achievement ones (emergency beliefs).

Influencing Power • Attractiveness, success, self-confidence, etc. • Coercive power • Manipulating power Etc. • Reliability • Credibility of BEL supporting GOAL • Competence of Source • Sincerity • Goal importance and risk Trust = BEL credibility * GOAL value .

(Social) Institutions.An Operational Notion • System explicitly designed or spontaneously evolved to regulate agents’ behaviour in a common environment. • Utilities: • Improve coordination and coperation • Produce externalities (including conventions) • Control common/collective resources • By means of norms and policies.

Institutional Influencing Power • Acknowledge-ability • Reliability • Empowerment (inter alia, to coerce)

Acknowledgement(From Conte, Castelfranchi, Dignum, 1998) • Input = a candidate norm (external norm). An obligation in the form OyX( q), q = the norm, y = authority that issues the norm and X = the set of the norm subjects. • Output = possibly a normative belief. Several tests: • evaluation of the c- norm: is it based on a recognised N? • evaluation of the source: Is agi entitled to issue N? This entails: • is q within the domain of y 's competence? • is the current context the proper context of q? • is X within the scope of y 's competence? • evaluation of the motives: is q issued for agi 's personal motives? The evaluation process is formalised as follows: • BELx(OzU( r)) & BELx(OzU( r)  OyX( q)) (10) • (OyX( q) & BELx(auth(y,X,q,C)) & BELx(mot(y,OK)))  BELx(OyX( q)) (11) Both lead to BELx(OyX( q)) The relation “auth”: y has authority to issue q on X in C. The relation “mot”: y's motives are correct.

Institutional Reliability What does it mean to trust institutions ?? • Institutional credit by definition • Circuits of institutional trust • Nested : • Trust a complex agent and therefore • its members • Competence • Disinterestedness (consequent to acknowledgement) • Tutoriality (“if she is a police-woman, she must be brave…”) • Transitive • Select • Empower • Control its representatives  Agents inherit institutional credit but may disconfirm it. • From institutional trust to goal importance (“if p is of institutional relevance it must be important…”), and from this back to trust.

Empowerment(cf. Sergot & Jones, 1995) • Contributes to • Acknowledgement • Trust • Direct influence • Coercion • Consists of • Resources (guns, gowns, etc.) • Action plan and procedures • Emergent properties of actions and plans (“With the power conferred by the Law, I hereby declare…”).

Acceptance(From Conte et al., 1998) • Is N-belief sufficient? No! Belief about instrumentality. • Normative corollary of social autonomy: x will form a N-goal q iff it believes that q is instrumental to a further goal: BELx(OyX( q)& INSTR(OBTX(q),p) &GOALx(p|r))  N-GOALx(OBTX(q)|GOALx(p|r) & r) (12) • Important differences from the g-generation rule: • the existence of a N-belief. But norms can be autonomously created: BELx(O(OyX( q)) & INSTR(OBTX(q),p)& GOALx(p|r))  N-GOALx(OBTX(q)|GOALx(p|r) & r)(13) • the form of the instrumental belief. But x may have internalised the norm: BELx(OyX( q) & INSTR(q,p) & GOALx(p|r))  C-GOALx(q|GOALx(p|r) & r) (14) • No N-conformity. We need: BELx(BELy(OzX( q)))  BELx(OzX( q)) (15) BELx(N-GOALy(OBTX(q)| r)  INSTR(OBTX(q),be_like(x,y))) (16) plus GOALx(be_like(x,y)|true) (17) • Otherwise “apparent N-adoption”!

So far... • Agents undergo social influence, that is they are often implicitly or explicitly requested to accept new goals. • Institutional influence is a special case of social influence. • In both cases, autonomous agents accept new goals (including normative ones) only as means to achieve old ones. • Questions • But what are the specific motives for accepting influence and forming new goals? • What is their respective efficacy? Which type of influence is more effective?

Norm (old) Goal (execute action) Motives for Acceptance Goal (old) Bel (p of connection) Goal (execute action) • Trust (probability/emergency belief) • Acknowledgement • Social Responsibility • Don’t harm • Material (e.g., passive smoking) • Symbolic harm (break institutional authority) • Don’t give a bad example Emotions Norm (acceptance) Bel (instrumentality) Bel (emergency)

Goal (avoid penalty Bel (instrumentality) Norm (acceptance) Goal (old) Side-Goal Bel (importance) Bel (importance) Goal (execute action) Meta-Goal Goal (old) Goal (execute action) Motives for Acceptance (cont’) Incentives • Negative • penalty • costs of action • obstacles • Positive • side-goals • meta-goals

Motives for Acceptance (cont’) Goal (old) • Social Control • Image and reputation • Responsible • Rational, consistent • Trustworthy • Social isolation • Social identity • Sharing (new) social norms & values Goal (accept influence) Bel (instrumentality) Goal (execute action) Bel (value or norm) Norm or Value (shared) Bel (instrumentality Goal (execute action

A Comparison Criteria (Dawkins, 1976) • Effectiveness • Fertility(range of influence) • Rapidity (how long it takes for fertility to reach threshold) • Stability • Fidelity (no copying-errors) • Transferability (Cavalli-Sforza & Feldmann, 1985) • Horizontal • Oblique • Vertical

Hypotheses

Preliminary Conclusions(to be checked by means of simulation) • Trade-off between social and institutional influence? • Social: low fidelity and low rapidity Vs fertility and transferability. • Institutional: high rapidity and fidelity Vs low fertility and transferability. • Which one is better? It depends on aims… (e.g., emergency requires policy-making). • Institutional influence should be supported by social control. This may provide guidelines for policy-making. For example, • Emotions or value-oriented? • Fear-inducing beliefs or responsibility? • New values (fitness, etc.) or side- and meta-goals? • But moreover, which specific type of influence is more effective than ohers: • trust-based Vs positive incentives Vs sanctions Vs reputation spread etc.

Maria Miceli Angelo Oddi Mario Paolucci  Roberto Pedone Giuliano Pistolesi  Paola Rizzo