1 / 24

Evaluation of a robot as embodied interface for Brain Computer Interface systems

Evaluation of a robot as embodied interface for Brain Computer Interface systems. Evaluation of a robot as embodied interface for Brain Computer Interface systems. L. Tonin, F. Piccione, S. Silvoni, K. Priftis, Emanuele Menegatti. K. Priftis Department of General Psychology

nelly
Download Presentation

Evaluation of a robot as embodied interface for Brain Computer Interface systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of a robot as embodied interface for Brain Computer Interface systems Evaluation of a robot as embodied interface for Brain Computer Interface systems L. Tonin, F. Piccione, S. Silvoni, K. Priftis, Emanuele Menegatti K. Priftis Department of General Psychology University of Padua, Italy IRCCS San Camillo, Venice, Italy Intelligent Autonomous Systems Lab University of Padua Dept. Of General Psychology University of Padua May 12, 2009 BCI Workshop ICRA 2009, Kobe

  2. Evaluation of a robot as embodied interface for Brain Computer Interface systems What is BCI? • According to Birbaumer et all.(1999): “Brain Computer interfaces are systems that allow to translate in real-time the electrical activity of the brain, in commands to control devices“ • Our target for the use of BCI system are patients severely paralyzed (locked-in) due to injury or disease • There are four general steps in a BCI system: • Signal acquisition • Classification and recognition • Interaction with device • Feedback to Subject May 12, 2009 BCI Workshop ICRA 2009, Kobe

  3. Evaluation of a robot as embodied interface for Brain Computer Interface systems Original P300-based BCI of IRCCS San Camillo • This BCI system is based on P300 wave detection • P300 is event-related potential • Visual interface gives to patient stimulus to activate P300 signal • The P300 detection will control the movement of cursor • Directional cues are represented by the single flash of one out four target-direction arrows • Participants focus their attention on the flashing arrow corresponding to the desired direction of motion • Validated on 5 tetraplegic patients and 7 healthy patients [Piccione et al. Clinical Neurophysiology 2006] May 12, 2009 BCI Workshop ICRA 2009, Kobe

  4. Evaluation of a robot as embodied interface for Brain Computer Interface systems P300-based BCI of IRCCS San Camillo Focusing own attention, patient will generate P300 signal: May 12, 2009 BCI Workshop ICRA 2009, Kobe

  5. Evaluation of a robot as embodied interface for Brain Computer Interface systems Goals of our project • In this project we aim at investigating two main issues: • Giving to locked-in patients the control of a physical actuactor to improve their rehabilitation experience • Developping telepresence of the patient with the body of the robot as medium • The questions are: • Can the robot lead patient to full and better control of BCI via a higher patient engagement in the task? • Might patients identify them-selves into the robot and think of it as an extension of their perceptions and actions (i.e. their body)? May 12, 2009 BCI Workshop ICRA 2009, Kobe

  6. Evaluation of a robot as embodied interface for Brain Computer Interface systems Robot as a physical actuator • To be able to move a body again, might give new strength and will to the locked-in patients • Custom built holonomous robot with hexagonal structure and three omnidirectional wheels • The holonomous nature of the robot is very important to replicate the motion and the apparence of the interface used by our BCI • The robot is fitted with: • Omnidirectional camera • Audio board • WiFi connectivity May 12, 2009 BCI Workshop ICRA 2009, Kobe

  7. Evaluation of a robot as embodied interface for Brain Computer Interface systems BCI and robot connection Robot Client Command BCI system SERVER Action result • Interaction between robot and BCI is based on client-server system with TCP-IP protocol via IEEE 802.11 • Security about connection between BCI and robot is guaranteed by: • System protocol and network protocol check data received; • Virtual Private Network between BCI and robot with autentication and encryption May 12, 2009 BCI Workshop ICRA 2009, Kobe

  8. Evaluation of a robot as embodied interface for Brain Computer Interface systems Test and Demo We carried out many tests to integrate BCI and robot. These tests accomplished at IRCCS San Camillo in Venice, and in IAS Laboratory of University of Padua: • Robot and BCI computer connected with wired network • Testing healthy subject performances to move robot • BCI and robot connected in larger network with VPN connection(robot was in Padova, and BCI system in Venice) May 12, 2009 BCI Workshop ICRA 2009, Kobe

  9. Evaluation of a robot as embodied interface for Brain Computer Interface systems Test and Demo During the scientific festival Discovery on Film in Rovereto (May 2008). A healthy subject at San Camillo in Venice was able to move the robot using the BCI at museum of Rovereto (about 200 Km from Venice). The test was successful and subject was able to reach the target. You can see it at: www.sperimentArea.tv the web TV of the Museo Civico of Rovereto May 12, 2009 BCI Workshop ICRA 2009, Kobe

  10. Evaluation of a robot as embodied interface for Brain Computer Interface systems Robot as actuator and generator of interface Robot is able to send back to patient images and sounds about environment.The images token by omnidirectional camera can be integrated in BCI interface to allow patient to see through robot’s camera.A new integrated interface for BCI is under clinical validitation by prof. Priftis of Psychology Department of University of Padua May 12, 2009 BCI Workshop ICRA 2009, Kobe

  11. Evaluation of a robot as embodied interface for Brain Computer Interface systems Robot as actuator and generator of interface Figure 6: classification accuracy (%) of the 5 healthy subjects who performed Experiment 1, and classification accuracy (%) of one subject who performed Experiment 2. May 12, 2009 BCI Workshop ICRA 2009, Kobe

  12. Evaluation of a robot as embodied interface for Brain Computer Interface systems Robot senses May 12, 2009 BCI Workshop ICRA 2009, Kobe

  13. Evaluation of a robot as embodied interface for Brain Computer Interface systems Robot as a Telepresence medium • The patient is able to see and listen with the senses of the robot and so he is able both to extend own social life and communicate own needs • Telepresence for museum visits: more autonomous navigation,obstacle avoidance, new graphical user interface • Telepresence for household rehabilitation: consumer robot Rovio holonomous robot with frontal camera for remote perception. May 12, 2009 BCI Workshop ICRA 2009, Kobe

  14. Evaluation of a robot as embodied interface for Brain Computer Interface systems • Robot as a Telepresence for museum visits • The patient is able to see and listen with the senses of the robot and so he is able both to extend own social life and communicate own needs The BCI Server TCP/IP Internet May 12, 2009 BCI Workshop ICRA 2009, Kobe

  15. The Client robots developed at University of Palermo, Sicily, ITALY Pioneer3-AT (Outdoor Robot) PeopleBot (Indoor Robot) ARES – CISIS 2009 Fukuoka Institute of Technology, Fukuoka 16 March 2009

  16. The Graphical Interface • STEP 1: Choose of the real robot ARES – CISIS 2009 Fukuoka Institute of Technology, Fukuoka 16 March 2009

  17. The Graphical Interface • STEP 2: The Session Start and Sensors and Actuators are Activated ARES – CISIS 2009 Fukuoka Institute of Technology, Fukuoka 16 March 2009

  18. The Graphical Interface • STEP 2.1: The Applet for the visualization of the navigation environment through the robot camera ARES – CISIS 2009 Fukuoka Institute of Technology, Fukuoka 16 March 2009

  19. The Graphical Interface • STEP 3: Robot Movements Activation through the use of the 4 directional arrows ARES – CISIS 2009 Fukuoka Institute of Technology, Fukuoka 16 March 2009

  20. The Graphical Interface ARES – CISIS 2009 Fukuoka Institute of Technology, Fukuoka 16 March 2009

  21. Evaluation of a robot as embodied interface for Brain Computer Interface systems • Robot as a Telepresence for museum visits • Next step: fully autonomous robot, robot has a map of the environment, robot presents four topological choices (e.g. the four closest masterpieces), subject selects destinations May 12, 2009 BCI Workshop ICRA 2009, Kobe

  22. Evaluation of a robot as embodied interface for Brain Computer Interface systems • Robot as a Telepresence for museum visits • Next step: fully autonomous robot, robot has a map of the environment, robot presents four topological choices (e.g. the four closest masterpieces), subject selects destinations May 12, 2009 BCI Workshop ICRA 2009, Kobe

  23. The Experimental Results: Simulation BCI- Web_Interface The BCI-Robot moving in the DINFO Department. BCI- Web_Interface Snapshot of Robot Position Snapshot of Robot Position Time T10: Command Stop Time T1: Command Go Forward

  24. Many Thanks to You and to: • Dept. of Information Engineering, University of Palermo, Palermo, Italy Rosario Sorbello: sorbello_rosario@unipa.it Antonio Chella: chella@unipa.it Salvatore Anzalone: anzalone@dinfo.unipa.it Francesco Cinquegrani: cinquegrani@dinfo.unipa.it • Dept. of Information Engineering, University of Padova, Padova, Italy Emanuele Menegatti: emg@dei.unipd.it • Dept. of Neuro-rehabilitation, S.Camillo Hospital, Venice, Italy Laboratory: labeeg.nfsancamillo@virgilio.it • Dept. of General Psychology, University of Padova, Padova, Italy Konstantinos Prifitis: konstantinos.priftis@unipd.it

More Related