The SNePS Approach to Cognitive Robotics

The SNePS Approach to Cognitive Robotics Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo shapiro@cse.buffalo.edu S.C. Shapiro

Outline • Introduction • Intensional Representation & Propositions • SNePS Connectives and Quantifiers • SNeRE Acting Constructs • Example Plans • Representation and Use of Indexicals • A Personal Sense of Time • Summary S.C. Shapiro

Goal • A computational cognitive agent that can: • Understand and communicate in English; • Discuss specific, generic, and “rule-like” information; • Reason; • Discuss acts and plans; • Sense; • Act; • Remember and report what it has sensed and done. S.C. Shapiro

Embodied Cassie • A computational cognitive agent • Embodied in hardware • or Software-Simulated • Based on SNePS and GLAIR. S.C. Shapiro

SNePS • Knowledge Representation and Reasoning • Propositions as Terms • SNIP: SNePS Inference Package • Specialized connectives and quantifiers • SNeBR: SNePS Belief Revision • SNeRE: SNePS Rational Engine • Interface Languages • SNePSUL: Lisp-Like • SNePSLOG: Logic-Like • GATN for Fragments of English. S.C. Shapiro

GLAIR Architecture Grounded Layered Architecture with Integrated Reasoning Knowledge Level NL SNePS Perceptuo-Motor Level Sensory-Actuator Level Vision Sonar Motion Proprioception S.C. Shapiro

Interaction with Cassie (Current) Set of Beliefs [SNePS] English (Statement, Question, Command) Reasoning Clarification Dialogue Looking in World GATN Parser (Updated) Set of Beliefs [SNePS] (New Belief) [SNePS] Answer [SNIP] Actions [SNeRE] GATN Generator Reasoning English sentence expressing new belief answering question reporting actions S.C. Shapiro

Cassie, the BlocksWorld Robot S.C. Shapiro

Cassie, the FEVAHR S.C. Shapiro

FEVAHR/Cassie in the Lab S.C. Shapiro

FEVAHRWorld Simulation S.C. Shapiro

UXO Remediation Cassie Corner flag Field Drop-off zone UXO NonUXO object Battery meter Corner flag Corner flag Recharging Station Cassie Safe zone S.C. Shapiro

Crystal Space Environment S.C. Shapiro

Entities, Terms, Symbols, Objects • Cassie’s mental entity: a person named Bill • SNePS term: B5 • Object in world: S.C. Shapiro

Intensional Representation Intensional entities are distinct even if coreferential. “The morning star is the evening star.” “George IV wondered if Scott was the author of Waverly.” S.C. Shapiro

McCarthy’s Telephone Number Problem Mary's telephone number is Mike's telephone number. I understand that Mike's telephone number is Mary's telephone number. Pat knew Mike's telephone number. I understand that Pat knew Mike's telephone number. Pat dialed Mike's telephone number. I understand that Pat dialed Mike's telephone number. S.C. Shapiro

Answering the Telephone Number Problem Did Pat dial Mary's telephone number? Yes, Pat dialed Mary's telephone number. Did Pat know Mary's telephone number? I don't know. S.C. Shapiro

Representing Propositions Propositions must be first-class entities of the domain Represented by terms. S.C. Shapiro

Discussing Propositions That Bill is sweet is Mary's favorite proposition. I understand that Mary's favorite proposition is that Bill is sweet. Mike believes Mary's favorite proposition. I understand that Mike believes that Bill is sweet. S.C. Shapiro

Logic for NLU &Commonsense Reasoning Either Pat is a man or Pat is a woman or Pat is a robot. I understand that Pat is a robot or Pat is a woman or Pat is a man. Pat is a woman. I understand that Pat is a woman. What is Pat? Pat is a woman and Pat is not a robot and Pat is not a man. S.C. Shapiro

Representation in FOPL? Man(Pat)  Woman(Pat)  Robot(Pat) S.C. Shapiro

Representation in FOPL? Man(Pat)  Woman(Pat)  Robot(Pat) but don’t want inclusive or S.C. Shapiro

+ + Representation in FOPL? Man(Pat)  Woman(Pat)  Robot(Pat) but don’t want inclusive or Man(Pat) Woman(Pat) Robot(Pat) T T T F T So don’t want exclusive or either S.C. Shapiro

andor andor(i, j){P1, ..., Pn} True iff at least i, and at most j of the Pi are True S.C. Shapiro

thresh thresh(i, j){P1, ..., Pn} True iff either fewer than i, or more than j of the Pi are True Note: thresh(i, j) ~andor(i, j) S.C. Shapiro

or-entailment {P1, ..., Pn} v=> {Q1, ..., Qn} True iff for all i, j Pi Qj S.C. Shapiro

and-entailment {P1, ..., Pn} &=> {Q1, ..., Qn} True iff for all j P1 &…& Pn Qj S.C. Shapiro

Numerical entailment {P1, ..., Pn} i=> {Q1, ..., Qn} True iff for all j andor(i, n){P1, …, Pn }Qj S.C. Shapiro

Universal Quantifier all(ū)({R1(ū),..., Rn(ū)} &=> {C1(ū),..., Cm(ū)}) Every ā that satisfies R1(ū)&…& Rn(ū) also satisfies C1(ū),..., Cm(ū)}) S.C. Shapiro

Numerical Quantifiers nexists(i,j,k)(x) ({P1(x),..., Pn(x)}: {Q(x)})} There are k individuals that satisfy P1(x) ... Pn(x) and, of them, at least i and at most j also satisfy Q(x) S.C. Shapiro

MENTAL ACTS • Believe(proposition) • Disbelieve(proposition) S.C. Shapiro

Act Selection • Do-One({act1 ... actn}) • Snif(if(condition, act), ... if(condition, act) [else(act)]) S.C. Shapiro

Act Iteration • Do-All({act1 ... actn}) • Sniterate(if(condition, act), ... if(condition, act), [else(act)]) • Snsequence(act1, ..., actn) • Cascade(act1, ..., actn) • P-Do-All({act1, ..., act2}) S.C. Shapiro

Entity Iteration WithSome(var, suchthat, do, [else]) WithAll(var, suchthat, do, [else]) WithSome+(var, suchthat, do, [else]) WithNew(vars, thatare, suchthat, do, [else]) S.C. Shapiro

Proposition/Act Transformers • Achieve(proposition) • ActPlan(act, plan) • GoalPlan(proposition, act) • Precondition(act, proposition) • Effect(act, proposition) • WhenDo(proposition, act) • WheneverDo(proposition, act) • IfDo(proposition, act) S.C. Shapiro

Conditional Plans If a block is on a support then a plan to achieve that the support is clear is to pick up the block and then put the block on the table. all(x, y) ({Block(x), Support(y), On(x, y)} &=> {GoalPlan(Clear(y), Snsequence(Pickup(x), Put(x, Table)))}) STRIPS-like representation: No times S.C. Shapiro

Use of Conditional Plan GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table))) Remember (cache) derived propositions. S.C. Shapiro

Use of Conditional Plan GoalPlan(Clear(B), Snsequence(Pickup(A), Put(A, Table)))??? SNeBR to the rescue! S.C. Shapiro

A FEVAHR Acting Rule all(p, o) ({Agent(p), Thing(o)} &=> {Precondition(Follow(p, o), Near(p, o)), GoalPlan(Near(p, o), Goto(p, o)), Precondition(Goto(p, o), Lookat(p, o)), ActPlan(Lookat(p, o), Find(p, o))}) Uses a temporal model. S.C. Shapiro

Acting According to the Rule S.C. Shapiro

Acting According to the Rule Follow a red robot. I found a red robot. I am looking at a red robot. S.C. Shapiro

Acting According to the Rule Follow a red robot. I found a red robot. I am looking at a red robot. I went to a red robot. I am near a red robot. I am following a red robot. S.C. Shapiro

A Plan for Blowing up UXOs all(a)(Agent(a) => ActPlan(Blowup(a, UXOs), Act(a, Cascade(SearchforUxo(a), WithSome+(obj, Near(a, obj), WithNew({ch ex}, {Charge(ch), Explosion(ex)}, Possess(a, ch), Cascade(Place(a, ch, obj), Hide(a), Waitfor(a, ex), SearchforUxo(a))), goto(a, SafeZone)))))) S.C. Shapiro

Representation and Use of Indexicals • Words whose meanings are determined by occasion of use • E.g. I, you, now, then, here, there • Deictic Center <*I, *YOU, *NOW> • *I: SNePS term representing Cassie • *YOU: person Cassie is talking with • *NOW: current time. S.C. Shapiro

Analysis of Indexicals(in input) • First person pronouns: *YOU • Second person pronouns: *I • “here”: location of *YOU • Present/Past relative to *NOW. S.C. Shapiro

The SNePS Approach to Cognitive Robotics