San Diego State University

1. San Diego State University College of Engineering A Web-Based Mobile Robotic System for Control and Sensor Fusion Studies Christopher Paolini1, Gerold Huber2, Quentin Collier3 and Gordon K. Lee1 1Dept of Elect &Comp Engr 5500 Campanile Drive San Diego State University San Diego, CA 92182 2Management Center Innsbruck University of Applied Sciences Universitätsstraße 15 6020 Innsbruck, Austria 3IUT de Bethune Networks & Telecomm Dept. 62408 Bethune, France

2. San Diego State University College of Engineering • Outline of Presentation: • The Mobile Robotic System Overview • The ANFIS Algorithm • Sensor Integration • Graphics User-Interface • Results • Conclusions and Future Work

3. San Diego State University College of Engineering Goal: Develop a mobile robotic testbed to investigate several control algorithms and sensor fusion techniques Approach: Use web-based video streaming and embedded control architecture for flexibility and robustness

4. San Diego State University College of Engineering

5. iRobot Create® Platform • Single Board Computer (SBC) • Linux Voyage • Unibrain Fire-i™ Digital Camera • Linksys Wireless-G PC adapter card • iRobot Create® platform • Single board computer (SBC) • Voyage Linux (Debian) • Unibrain Fire-i™ digital camera (IEEE 1394) • Proxima 802.11g PCMCIA adapter card with external 5db gain antenna

6. San Diego State University College of Engineering iRobot Create® with Sensor Arrays

7. Experimental iRobot developed at San Diego State University Unibrain Fire-i™ Digital Camera • iRobot Create® based robot designed with several sensors • Streaming video • Web teleoperation • ANFIS automation 2.5dBi gain indoor omni-directional antenna IR Sensor Thermal Sensor Arduino Mega MCU 9DOF Inertial Measurement Unit Migrus C787 DCF-P single board computer with a 1.2GHz Eden ULV Processor 360° Ultrasonic Sensor Array 802.11g PCMCIA Transceiver

8. San Diego State University College of Engineering Sensors 3-Axis Magnetometer 3-Axis Rate Gyroscope 3-Axis Accelerometer Inertial Measurement Unit Controller

9. San Diego State University College of Engineering Ultrasonic sensor array using an Arduino Mega 2560 microcontroller

10. San Diego State University College of Engineering Ultrasonic Sensor Array MaxBotix LV-EZ1 Ultrasonic Sensor An array of 10 MaxBotix LV-EZ1 sensors suspended on two circular plates. Each MaxBotix sensor provides a 36 degree FOV.

11. San Diego State University College of Engineering Thermal Sensor

12. How the iRobot Adjusts Its Heading • The iRobot changes its azimuth by sending a 16 bit signed value in the range [-2000, 2000] mm that defines a turning radius • The turning radius is a ray from the center of the turning circle to the center of the robot • r > 0  robot turns left • r < 0  robot turns right • Special cases: r = 32768 or 32767 (0x8000 or 0x7FFF) causes robot to move straight • r = 0xFFFF  robot turns in place clockwise • r = 0x0001  robot turns in place counter-clockwise r1 r2 Large radius  small curvature Small radius  large curvature

13. How the Web GUI Determines the Turning Radius (x,y) In quadrant I and IV for >5 II I In quadrant II and III for <-5  IV III Assume r varies linearly with 

14. Inertial Measurement Unit (IMU) • Robot has an onboard 9DOF IMU • Incorporates four sensors: LY530ALH single-axis gyro, LPR530AL dual-axis gyro, ADXL345 triple-axis accelerometer, and a HMC5843 triple-axis magnetometer • Gives nine degrees of inertial measurement • LY530ALH: STMicroelectronics ±300 °/s analog yaw-rate gyroscope • HMC5843: Honeywell HMC5843, a 3-axis digital magnetometer outputs Euler X,Y,Z orientation vectors and roll () and pitch () angles (tilt sensor) with 12-bit ADC at 10 Hz • MCU computes azimuth or “yaw” with an accuracy of 1-2 º

15. How to Model Robot Dynamics while Turning? • We want the absolute bearing defined by the remote user with the GUI to equal the absolute bearing reported by the IMU • How to compensate for unknown system dynamics? • Can define a neural network to model robot (plant) dynamics and use training data to tune network parameters • Define a set of 4-tuple training data: which are the ith desired (from GUI) azimuth, azimuth rate, actual (from IMU) azimuth and azimuth rate, respectfully. From the HMC5843 digital magnetometer From the LY530ALH yaw rate sensor

16. San Diego State University College of Engineering ADAPTIVE AJAX-BASED STREAMING VIDEO SYSTEM AJAX based Web interface for telerobotic control of the iRobot Create Encoding bit rate as a function of fps

17. Virtual Force Field Approach for Obstacle Avoidance

18. The whiteness of the cell is proportional to the CV value The Certainty Grid • 2-Dimensional array of cells • Each cell contains a Certainty Value (CV) • CV indicates the measure of confidence that an object exist within a cell • Instantaneous map for obstacle representation • dx = dy = 15 cm • 21 by 21 square cells represent the Certainty Grid Front

19. Method for Updating Certainty Values Each sensor corresponds to a particular angle Ө, based on its position on the sensor assembly At a given time, a sensor returns a distance d Eq. 1 and 2 transform (d, Ө) → (x’,y’) (1) (2)

20. Obstacle Avoidance for Path Planning Task Port Side Front

21. San Diego State University College of Engineering Cells Located on the Acoustic Axis

22. San Diego State University College of Engineering (a) Histogram Grid ; (b) Snapshot of Video Camera

23. San Diego State University College of Engineering The ANFIS Architecture If xis Ai and y is Bj, then

24. San Diego State University College of Engineering Off-line Training Forward pass: consequent parameters Backwards pass: premise parameters

25. San Diego State University College of Engineering On-line Learning Forward pass: consequent parameters Backwards pass: premise parameters

26. San Diego State University College of Engineering

27. San Diego State University College of Engineering Simulation Results

28. System Integration through an Arduino MCU • Multiple sensors: thermal (person/fixture differentiation), ultrasonic (collision avoidance and path planning), IR (automatic docking), video (telerobotic control), and magnetometer and accelerometer (orientation and position) have been tested and integrated into the overall system architecture. • ANFIS controller is being implemented on an Arduino Mega that uses the magnetometer and accelerometer output for path tracking. • Initial simulation studies have been conducted using a MISO controller to test this proof of concept. • Then performed actual experimentation using one and two inputs to the ANFIS controller

29. Initial Web Browser Interface

30. San Diego State University College of Engineering

31. San Diego State University College of Engineering Experimental Results Bearing Scenario

32. San Diego State University College of Engineering Bearing Scenario

33. San Diego State University College of Engineering Constant Turn

34. San Diego State University College of Engineering Following Scenario

35. San Diego State University College of Engineering Learning Scenario

36. San Diego State University College of Engineering ANFIS Responding Time

37. San Diego State University College of Engineering Conclusions and Future Work • A MIMO ANFIS controller has been designed and tested through simulation and experimental studies • The desired controller can adaptively adjusting to system variations through supervised and un-supervised learning. • Future tasks include extending the MIMO design to a multiple inputs, two output structure and evaluate the performance of this MIMO implementation using Player/Stage and experimentation. • We will add on-line learning functionality to our embedded MIMO ANFIS that will effectively tune the parameters computed from off-line training data.

38. San Diego State University College of Engineering

39. San Diego State University College of Engineering