Hani M. Sallum Masters Thesis Defense May 4, 2005

1. Development and Implementation of a High-Level Command System and Compact User Interface for Nonholonomic Robots Hani M. Sallum Masters Thesis Defense May 4, 2005

2. Outline • Overview and Goals • Development • Control System • Data Analysis and Mapping • Graphical User Interface • Results

3. Overview This work details the design and development of a goal-based user-interface for unmanned ground vehicles which is maximally simple to operate, yet imparts ample information (data and commands) between the operator and the UGV.

4. Typical UGV Usage

5. Other UGV Issues • Multi-person crews • Proprietary Operator Control Units (OCU’s) for each UGV Is there a way to have local users control UGV’s without the operational overhead currently required?

6. Current State of the Art

7. Why UA/GV’s? • Over the last two decades there has been a dramatic increase in the complexity/availability of manufactured electronics. • As a result, the capital cost of robotic systems in general has decreased, making them more feasible to implement. Example: N.A.S.A. Mars Pathfinder/Sojourner system was built largely out of commercially available (OTS) parts (sensors, motors, radios, etc.)1. • Additionally, the capacity and functionality of devices such as PDA’s and cellular phone has increased as well. 1. N.A.S.A., Mech. Eng. Magazine, Kodak

8. Motivation: Q: Do custom O.C.U.’s need to be developed when commercial technology is evolving so rapidly? Considering the ubiquity of PDA’s, smartphones, etc., is it possible to develop a method of using these devices as a form of common O.C.U.?

9. Goals: • Develop a control system for a UGV • Automates low-level control tasks • Develop a method of rendering sensor data into maps of the UGV’s environment • Design a GUI which runs on a commercially available PDA

10. Camera Sonar Laser Rangefinder Hardware: Robot iRobot B21R Mobile Research Robot (nonholonomic)

11. Definition of Nonholonomic Unable to move independently in all possible degrees of freedom. Example: Cars have 3 degrees of freedom (x,y, q), but can not move in x or q alone.

12. 802.11/BluetoothAntenna Windows CE Operating System Hardware: PDA Hewlett-Packard iPAQ 240x320 Color Screen Touch Sensitive

13. Navigation Control System Two aspects of the navigation process: • Target Approach • Obstacle Avoidance Multimodal Controller Separate control laws depending on the desired operation of the robot.

14. Proven Method • Schema Architecture [Chang et al.] • Discrete shifts between control modes • Straightforward to implement • “chattering” between modes

15. Proposed Method • Fuzzy Control [Wang, Tanaka, Griffin] • Gradual shifts between control modes • More complicated controller • Smoother trajectory through state-space

16. Fuzzy Controller Sensor Data Target Approach Mode Obstacle Avoidance Mode {K,w} {K,w} Fuzzy Blending Fuzzy Control Signals {v,w}

17. Target Approach Control of Turning Velocity • Final orientation unconstrained • Implement a proportional controller driving the robot heading to a setpoint equal to the current bearing of the target (i.e. qDEV  0) • Produce wAPP • Saturate the controller at the max allowable turning speed • Use high proportional gain to approximate an arc-line path

18. Target Approach Control of Forward Velocity • Final position close to target • Implement a proportional controller to scale the forward velocity, based on the robot’s distance to the target coordinates (i.e. DTAR  0) • Produce KAPP • Saturate the controller at KAPP =1 (scale to the maximum allowable forward speed)

19. Obstacle Avoidance Control of Turning Velocity • Implement a proportional controller driving the robot heading to a setpoint 90º away from the nearest obstacle (i.e. qOBS  ±90º) • Produce wAVOID

20. Obstacle Avoidance Control of Forward Velocity • Implement a proportional controller to reduce (scale down) the forward velocity when nearing an obstacle • Produce KAVOID

21. Obstacle Avoidance • Forward Control • Inner threshold elliptical to avoid being stuck to obstacles:

22. Final Control Law Turning Control: • Blend the target approach and obstacle avoidance control signals using a weighted sum: WAPPwAPP + WAVOIDwAVOID = wFUZZY • Determine weights using membership functions based on the robot’s distance to the nearest obstacle.

23. Final Control Law Forward Control: • Blend the target approach and obstacle avoidance control signals by multiplying the maximum forward velocity by the scaling factors produced by each control mode. KAPPKAVOIDvMAX = vFUZZY

24. Outline • Overview and Goals • Development • Control System • Data Analysis and Mapping • Graphical User Interface • Results

25. Data Analysis and Mapping • Render data from the laser rangefinder into significant features of the environment: “Fiducial Points” e.g. corners, ends of walls, etc. • Use these fiducial points to generate primitive geometries (line segments) which represent the robot’s environment.

26. Object Detection Segment Detection Finding Intersections Line Fitting Categorizing Points FIDUCIAL POINTS Distillation Process Finding Fiducial Points RAW DATA

27. Why Find Fiducial Points? Laser Rangefinder Data

28. Object Detection Range vs. Bearing (used by Crowley [1985])

29. Object Detection

30. Object Detection

31. Segment Detection Recursive Line SplittingMethod used by Crowley [1985], B.K. Ghosh et al. [2000]

32. Segment Detection Proposed Threshold Function CREL: Relative Threshold CABS: Absolute Max Threshold

33. Line Fitting Use perpendicular offset least-squares line fitting

34. Finding Intersections

35. Categorization • Each fiducial point is either interior to an object, or at the end of an object. • Fiducial points at the ends of objects are either occluded or unoccluded.

36. Distillation Finding Fiducial Points

37. Mapping • Fiducial Points provide a clear interpretation of what is currently visible to the robot • Provide a way to add qualitative information about previously observed data to local maps Global map

38. Creating a Global Map Global Map:Occupancy Evidence Grid [Martin, Moravec, 1996] based on laser rangefinder data collection.

39. Creating a Global Map Global Map:Occupancy Evidence Grid [Martin, Moravec, 1996] based on laser rangefinder data collection.

40. Local Mapping Map Image: • Sample section of global map for qualitative a priori information about local area. • Overlay map primitives.

41. Local Mapping Example: Local map with and without a prior information.

42. Vision Mapping Vision Map: • Transform map primitives to perspective frame and overlay camera image of local area.

43. Vision Mapping Find common geometries for defining vertical and horizontal sight lines.

44. GUI • Serve web content from the robot to the iPAQ • Use image-based linking (HTML standard) to allow map images to be interactive on the iPAQ • Use web content to call CGI scripts onboard the robot, which run navigation programs on the robot

45. GUI Main Map/Command Screen

46. GUI Close-Range Map/Command Screen Long-Range Map/Command Screen

47. GUI Vision Map/Command Screen Rotation Map/Command Screen

48. Results • GUI: Main Map/Command Screen

49. Results • GUI: Rotation Map/Command Screen

50. Results • GUI: Vision/Command Screen