590 likes | 599 Views
WP1: Action Development. V. Tikhanoff 1,2 , F Nori 2 ,G Metta 2 , A. Gijsberts 2 , A. Cangelosi 1 D. Marocco 1 , S. Nolfi 3 1 Adaptive Behaviour and Cognition, University of Plymouth (UK) 2 Robotics Brain & Cognitive Sciences, Italian Institute of Technology (Italy)
E N D
WP1: Action Development V. Tikhanoff1,2, F Nori2,G Metta2, A. Gijsberts2, A. Cangelosi1 D. Marocco1, S. Nolfi3 1 Adaptive Behaviour and Cognition, University of Plymouth (UK) 2 Robotics Brain & Cognitive Sciences, Italian Institute of Technology (Italy) 3 Institute for Cognitive Science and Technology, CNR, Rome (Italy) ITALK Year 1 ReviewDüsseldorf, 30 June 2009
WP1, goals • To design and develop an enhanced iCub robotic platform • To develop an iCub simulation application • To carry out robotic experiments on the development of object manipulation • To develop a motor controller “compatible” with language development, i.e.: • Compositional • General • Hierarchical • To investigate how to develop complex compositional and hierarchical actions ITALK Year 1 ReviewDüsseldorf, 30 June 2009
WP1, tasks • Task 1.1: Development of extended iCub platform for action manipulation and language learning experiments • Task 1.2: Development and test of iCub simulation software for multi-agent scenarios • Task 1.3: Development of object manipulation capabilities • Task 1.4: Experiments on compositional and hierarchical actions • Task 1.5: Interaction and integration between multiple motor skills ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Task 1.1, iCub and the construction of multiple copies ITALK Year 1 ReviewDüsseldorf, 30 June 2009
The iCub: quick summary • The iCub is the humanoid baby-robot designed as part of the RobotCub project • The iCub is a full humanoid robot sized as a three and half year-old child. • The total height is 104cm. • It has 53 degrees of freedom, including articulated hands to be used for manipulation and gesturing. • The robot will be able to crawl and sit and autonomously transition from crawling to sitting and vice-versa. • The robot is GPL/FDL: software, hardware, drawings, documentation, etc. ITALK Year 1 ReviewDüsseldorf, 30 June 2009
iCub “size” • Three large assemblies • Head • Upper body • Lower body • More than 120 sub-assemblies • 230 commercial codes for a total of 2967 parts • 394 custom-made codes for a total of 1110 parts • Electronics, 200 components per card (on average): approximately 5000 additional parts • Each robot is made of more than 9000 parts ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Degrees of freedom • Head: vergence, common tilt + 3 dof neck • Arms: 7 dof each • Shoulder (3), elbow (1), wrist (3) • Hands: 9 dof each ► 19 joints • 5 fingers ► underactuated • Legs: 6 dof each • Hip (3), knee (1), ankle (2) • Waist: 3 dof Σ= 53 dof (not counting the facial expressions) ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Sensorization • Absolute position • On most joints, AMS magnetic encoder • Cameras • Pointgrey Dragonfly firewire cameras • Microphones, speaker • Standard condenser electrect miniature microphones • Pinnae • Gyroscopes, linear accelerometers • Xsense: Mtx ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Custom electronics • ADC card • Special connectors (40 pins < 1cm length) • 200μm stainless steel wires, coated in Teflon • Motor control • C programmable DSP 40 MIPS • Up to 4A DC motor 80x30mm Motorola DSP56F807 (5680x family) MAC instructions PWM generation ADC Digital I/O Can bus C programmable 58x42mm ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Wiki CVS Part lists Drawings
By spring 2010 we will have 18 working platforms ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Promoting the iCub • RobotCub Open Call • 31 participants, 7 winners will receive a copy of the iCub free of charge • UPMC Paris, Imperial London, Inserm Lyon, TU Munich, METU Ankara, Pompeu Fabra Barcelona, Urbana-Champaign USA, IST Lisbon, EPFL Lausanne • Further development… • EU project ITALK: 4 iCub’s have been built • EU project ImClever: 3 iCub’s will be built • EU project RoboSkin: a skin system compatible with iCub • EU project CHRIS: safety features for the iCub • Collaborations • Univ. of Karlsruhe: new and longer legs • Simulator: • Open Source simulator based on ODE and Newton and as a model in Webots ITALK Year 1 ReviewDüsseldorf, 30 June 2009
In the pipeline… • Force control: joint level sensors, SEA or stain gauges based sensing • Skin/tactile sensors: almost everywhere on the robot surface • Robot general improvements: e.g. zero-backlash everywhere, better control electronics, higher resolution position sensors ITALK Year 1 ReviewDüsseldorf, 30 June 2009
The skin Principle Lot of sensing points Structure of the skin
Fingertips • Capacitive pressure sensor with 12 sensitive zones • 14.5 mm long and 13 mm wide, sized for iCub • Embedded electronics: twelve 16 bit measurements of capacitance • either all 12 taxels independently at 50 Hz or an average of the 12 taxels at about 500 Hz ITALK Year 1 ReviewDüsseldorf, 30 June 2009
6-axial force/torque sensor • Semiconductor strain gauges • On board signal conditioning, sampling, and calibration • Digital output: CAN bus • Design: Nikos Tsagarakis and Darwin Caldwell (IIT) • Electronics: Claudio Lorini (IIT) ITALK Year 1 Review Düsseldorf, 30 June 2009
The Software • Goals: • Foster collaboration in “space” and “time”… • … since we’re a relatively large community and we don’t want to re-invent the wheel too often • Manage the complexity of the hardware… • … since humanoid robots (like the iCub) are complicated • Consequence: improved on existing Open Source libraries supporting a major overhaul of YARP to the iCub ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Sensors Cluster Relay station (head) DSP PC1 DSP Ethernet HUB DSP Gbit PC104 DSP PCN Actuators iCub API Low-level control Yarp Embedded iCub structure higher level capabilities control of attention reaching grasping learning imitation … … ITALK Year 1 ReviewDüsseldorf, 30 June 2009
YARP is an open-source middleware for humanoid robotics • History • An MIT / Univ. of Genoa collaboration • Born on Kismet, grew on COG • With a major overhaul, now used by RobotCub consortium • Exists as an independent open source project • C++ source code • In short: it’s the plumbing ITALK Year 1 ReviewDüsseldorf, 30 June 2009
It factors out: • the data flow: inter-process communication • it is often useful to keep algorithms away from the plumbing • the hardware: device drivers model • it is useful to avoid references to the hardware in the source code • …while being portable: • across OS and development tools • across languages • libs in C++, bindings for many other languages ITALK Year 1 ReviewDüsseldorf, 30 June 2009
The iCub With Peter Ford-Dominey (INSERM, Lyon) With Auke Ijspeert, Ludovic Righetti, Sarah Degallier (EPFL) With a lot of students @ RobotCub summer school 2008 With VisLab (IST Lisbon)
iCub platform extensions • Software: • definition of a yarp-portable audio data format; • design of a portaudio library wrapper; • design of a ASR (automatic speech recognition) interface; • integration of ASR software (Sphinx and Esmeralda). • Hardware: • facial expressions hardware standardization and production (installed in Bielefeld); • microphones integration (installed in Bielefeld); • stereo line-in amplifier integration (installed in Bielefeld). ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Hardware extensions • Face expressions: • testing; • hw standardization; • production. • Integrated microphones: • electret condenser; • functional pinnae; • custom amplifiers. • Enhanced PCI card: • 4 CAN buses interfaces; • 2 firewire; • stereo line-in amplifiers. ITALK Year 1 Review Düsseldorf, 30 June 2009
Software extensions • yarp-portable audio format: • yarp:sig:sound • yarp-wrapper for libportaudio: • yarp:dev:PortAudioDeviceDriver ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Speech Recognition Engine Integration • CMU SPHINX: • a group of open source systems related to speech recognition; • speech recognition systems developed at Carnegie Mellon University. These includes a series of speech recognizers (Sphinx 2 - 4) and an acoustic model trainer (SphinxTrain). ITALK Year 1 ReviewDüsseldorf, 30 June 2009
YARP/ASR interface design Microphone (windows version) yarp:sig:sound yarp:sig:sound Microphone (linux version) Segment “Get a sound” Interface (IAudioGrabberSound) “Get a sound” Interface (IAudioGrabberSound) Decode, DAG-search PortAudio library interface Cepstral Analysis Classify According to acoustic model type Acoustic Adaptation FFMPEG Grabber library interface Cluster Decode, DAG-search N-Best Reformat & Collate N-Best rescore Words ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Task 1.2, simulator ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Cognitive Robotics & Simulators • Why do we need a robot simulator? • Not an alternative to physical robot, but an additional tool; • preliminary testing/feasibility studies; • experimenting alternative configurations; • to ease accessibility to robot and collaboration within and outside the italk consortium. • Existing robot simulation applications: • Stage / Gazebo Project ( 2D / 3D multiple-robot simulator); • Simbad Project (3D multiple-robot simulator); • Darwin 2K Project (Simulation & Automated Synthesis for Robotics); • EvoRobot (Evolutionary Robotics Simulator); ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Physics & rendering engines • A Physic Engine library provides: • Simulation of rigid bodies (mass, friction, sensors etc..) • Collision Detection Algorithms for all the physical interactions • Advanced joint types (also customizable) • Terrains and/or Meshes for complex object creation • There are many available physic engines with their pro and cons • Different solvers for rigid-body dynamic • Different spaces for collision detection • More joint types already implemented • OpenSource, GPL license • Multi-thread capability • More accurate friction simulation • API for custom joint creation and control • Free license but not OpenSource Robot Simulation – Robot Programming with Open Dynamics Engine Morikita Publishing Co. Ltd Tokyo 2007 ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Physics engine & rendering • WorldSim library has been created for providing • Support different physic engines • Object-Oriented design for an easy creation of a custom version of simulator • Possibility to simulate more than one iCub • 3D renderiomg classes based on OpenGL ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Software Architecture • Objective: control robot with same commands for both real & simulated iCub: • use the same software infrastructure and inter-process communications. • Tools: real and simulated robot software architecture is based on YARP: • motor interface: yarp::dev::controlBoard; • video interface: yarp::sig::Image; • sensor interface: yarp:os:Portable. ITALK Year 1 ReviewDüsseldorf, 30 June 2009
The iCub Platforms • iCub Physical Robot and iCubSim Simulated robot • Total height 105 cm • Total weight 20.5 kg • 53 DoFs in Total • 12 for the legs • 3 for the torso • 32 for the arms • 6 for the head • Emotion interface • Touch sensors ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Robot Control ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Transferability – Simulator to Physical • Requires just switching the robot name in the configuration files or programs from “icubSim” to “icub”. • The simulated robot shares some important robot features: • kinematic specifications; • motor interface (controlBoard interface) ; • vision. • The simulated robot is not a “perfect” match of the physical robot: • rough (body) dynamical model; • rough trajectory generator; • rough actuators dynamical model. ITALK Year 1 ReviewDüsseldorf, 30 June 2009
iCubSim demo • Launching the simulator. • Accessing the motor interface: • via command line (yarp rpc); • via the robotMotorGui; • via C++ code (preprogrammed sequences). • Accessing the video interface: • via the yarpview. • Object manipulation scenario. ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Task 1.3, motor (and cognitive) capabilities ITALK Year 1 ReviewDüsseldorf, 30 June 2009
On-line life-long learning • Developmental robotics entails: • Quasi online • at least memory bounded • adapt to changes in the dynamics (e.g. tools) • Fast computation of the solution • useful in control loops • Large data-sets • perhaps collecting data for the entire “life” of the robot • Guarantees • convergence, bounds, etc. ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Methods • Kernel methods • Online variants • More of a research area • ANN, feedforward, recurrent • Training via backpropagation • Via reinforcement • More established ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Scenario: dataset • Inputs: Joint positions, velocities and accelerations for 4 joints (3 shoulder, 1 elbow) • Outputs: Forces and torques in x, y, z directions (here we only consider Fx) • 100.000 samples collected at 50 Hz during random movements • Inputs scaled “roughly” within [-1,+1] ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Some batch results ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Passive Aggressive regression • Variant of Perceptron algorithm • Margin-based (also updates with margin errors) like SVM • Easily kernelized using kernel expansion • Binary and multiclass classification, regression, etc. • Efficient and easy to implement Crammer et al., Online passive-aggressive algorithms, 2006. ITALK Year 1 ReviewDüsseldorf, 30 June 2009
On-line, efficient, bounded Time vs. # of samplesexplicit polynomial Time vs. # of samples forpolynomial kernel Accuracy for randomapproximation Time vs. # of samples forrandom approximation ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Learning motor control • Map sensory information into appropriate behaviors: • Gazing • Reaching • Grasping ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Example: gazing ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Reaching • Model acquisition for reaching ITALK Year 1 ReviewDüsseldorf, 30 June 2009
Task-space control Major drawbacks of available algorithms for IK (Pseudoinverse, DLS): need to manage singularities, need of boundary functions to embody the notion of joints limits.
Example Comparison of random XYZ positions of objects, with the actual resulting position of the robots hand ITALK Year 1 ReviewDüsseldorf, 30 June 2009