Intelligent Robotics

1. Intelligent Robotics Jeremy Wyatt Thanks to: Nick Hawes, Aaron Sloman, Michael Zillich, Somboon Hongeng, Marek Kopicki

2. The Whole Iguana • AI commonly studies aspects of intelligence separately: narrow domain high performance • In 1976, philosopher Dan Dennett suggested putting it all together, but with a low level of performance • In fact people had been trying to build integrated systems for some twenty years by then

3. Shakey the robot • 1970 - Shakey the robot reasons about its blocksBuilt at Stanford Research Institute, Shakey was remote controlled by a large computer. It hosted a clever reasoning program fed very selective spatial data, derived from weak edge-based processing of camera and laser range measurements. On a very good day it could formulate and execute, over a period of hours, plans involving moving from place to place and pushing blocks to achieve a goal. (Hans Moravec)

4. 30 20 10 0 r2 f4 f3 o1 d2 r3 f2 f1 d1 shakey r1 0 10 20 Shakey: key ingredients • World model used logical representations type(r1,room) in(shakey,r1) in(o1,r2) type(d1 door) type(o1 object) type(f3 face) type(shakey) at(o1 15.1 21.0) joinsfaces(d2 f3 f4) joinsrooms(d2 r3 r2) …

5. 30 20 10 0 r2 f4 f3 o1 d2 r3 f2 f1 d1 shakey r1 0 10 20 Planning • Shakey used a form of planning called goal regression • Idea: find an action that directly achieves your goal, and then actions to achieve the first action’s preconditions, etc… • e.g. Blocked(d1,X) • Let’s see Shakey solve a problem block_door(D,Y) preconditions: in(shakey,X) & in(Y,X) & clear(D) & door(D) & object(Y) delete list: clear(D) add list: blocked(D,Y)

6. Lessons from nature • Gannets – wings half open to control dive • Fold wings to avoid damage • Not at a constant distance, but at a constant time • Birds have detectors that calculate time to impact

7. Lessons from nature • All naturally occuring intelligence is embodied • So robots are in some ways similar systems • Robots, like animals exploit their environments to solve specific tasks “There are no general purpose animals … why should there be general purpose robots?” David MacFarland

8. Behaviour Based Robots • Inspired by simpler creatures than humans • Throw away most representations • Throw away most reasoning • Build your robot out of task specific behaviours

9. Pushing the behaviour based envelope • Behaviour based systems can display quite sophisticated behaviour, particularly for interaction • But they don’t have understanding because they don’t have representations

10. The age of data • In the 1990s people were finally beginning to have success with representation driven approaches • One key has been the use of probabilistic methods • These are data intensive and require very strong assumptions about the learning task • Stanley

11. B1:phys-object ^ ball <property> C1:colour ^ orange Robots that understand • Internal structure to represent the meaning of the utterance e.g. “The orange ball”

12. Learning object appearances

13. Learning names and appearances of objects

14. Cognition requires attention • Object recognition is unreliable and expensive • We can use bottom up salience to make it more efficient

15. Directing processing of the visual scene Salience can be modulated by language

16. Manager Sensor Actuator Processor Working Memory The Whole Iguana: coming full circle • Collection of loosely coupled sub-architectures • Each sub-architecture contains processing elements that update structures within a working memory • WM are typically only locally read & write (bar privileged sub-architectures) • Processing controlled by local and global goals and managers • Knowledge management by local and global ontologies

17. Illustration: Cross Modal Ontology Learning Architectures Movie goes here

18. Communication SA Communication SA Communication SA Communication SA Visual SA Planning SA Manipulation SA Communication SA Binding SA Spatiotemporal SA Coordinator SA Communication SA Communication SA Communication SA Illustration: Language Driven Manipulation Architectures Linguistically Driven Manipulation • Goals are raised by language • Reference to objects by learned features • Robot plans intentional actions using logical planner • Intention shifting is handled via execution monitoring and continual replanning

19. “Put the blue thing to the right of the red thing” Parse + Dialogue Interpretation Execution Check Qual Spatial Relations Qual Spatial Relations Qual Spatial Relations Object locations Object locations Object locations Inst 2 Inst 2 Inst 2 Inst 1 Inst 1 Inst 1 SO 2 SO 2 SO 1 SO 2 SO 1 SO 1 Vis Servo Raise Planning Goal Goal LF ROI 2 ROI 2 ROI 1 ROI 1 Raise Manip Goal Manip Goal Executed Plan Step ROI 2 ROI 1 Plan MAPL Goal Communication SA Coordination SA Manipulation SA Planning SA Spatial SA Binding SA Visual SA

20. Illustration: Language Driven Manipulation Architectures Movie goes here

21. Wrap up • Robotics gets to the heart of big issues in AI • There are enormous engineering and scientific challenges • There is a tension between different approaches: • Representation heavy, top-down approaches to cognition • Representation light, bottom approaches • The fun is in linking these