1 / 54

CBWare - Distributed Middleware for Autonomous Ground Vehicles

This Master's thesis presents CBWare, a distributed middleware for autonomous ground vehicles, focusing on real-time information exchange, sensor data fusion, log server, and remote real-time monitoring. The thesis discusses motivation, research contributions, background, related work, evaluation results, conclusions, and future work.

angelbrown
Download Presentation

CBWare - Distributed Middleware for Autonomous Ground Vehicles

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CBWare - Distributed Middleware for Autonomous Ground Vehicles Master’s Thesis Defense By Vidhyalakshmi Venkitakrishnan Advisor: Dr. Arun Lakhotia Center for Advanced Computer Studies University of Louisiana at Lafayette October 18, 2006

  2. Presentation Outline • Part I - Motivation and Research Contributions • Part II - Background and Related Work • Part III - CBWare and Evaluation Results • Part IV - Conclusions and Future Work

  3. Middleware in AGV’s - Motivation

  4. I. Real-time Information Exchange • Processes are computationally intensive and complex • Need for distributing components on multiple machines to achieve fairness and efficiency • Middleware – Key to information exchange among distributed components

  5. CajunBot

  6. II. Sensor Data Fusion LIDAR (Range, Theta) INS (x, y, z) Global (x, y, z)

  7. Sensor Data Fusion – contd. • LIDAR Queue • INS Queue • Fusing most recent reading from queues – not consistent • Both sources generate data at different frequencies

  8. CajunBot on rough terrain DARPA GC 2005

  9. Sensor Data in Rough Terrain t1

  10. III. Log Server • Need data from processes for post-analysis • Collect data to tune parameters • CajunBot – NQE 2005 • Z spike - wall of obstacles detected from log data

  11. IV. Remote Real-time Monitoring

  12. Why Real-time monitoring? • Visual/graphical views of the world seen by system • Internal states of Obstacle Detection • Visualize live data during field testing • Debug problems with processes in real-time

  13. Research Contributions • Real-time Information Exchange in distributed system • Support for fusion of consistent data • Log server • Real-time monitoring capability

  14. What is Middleware? Layer of software that connects different application components. Supports complex, distributed applications Hides heterogeneity of underlying platforms Middleware Background

  15. Types of Middleware • Publish/Subscribe • Client/Server • Remote Procedure Calls • Other IPC Mechanisms: Shared Memory, Message Queues, UNIX IPC

  16. Publish/Subscribe Model Subscriber subscribes to data Publish/Subscribe Middleware Publisher Subscriber Publisher publishes data Middleware delivers data • Powerful abstraction for distributed systems • Message-based anonymous communication • Publishers and Subscribers are decoupled • Good solution for scalability • Examples: CMU-IPC, NDDS, IBM-Gryphon, Siena

  17. Client/Server Model Client 1 Request Client 3 Server Response Client 2 • Clients send request to the server • Server processes requests and responds back • Clients blocks until response from server

  18. Remote Procedure Calls • Protocol to extend Local Procedure • Involves two independent processes, which may reside on different machines • Caller (Process A on Host A) issues procedure call to Callee (Process B on Host B) with list of argument values • Caller is suspended • Callee executes procedure, returns values to Caller • Caller resumes execution • Examples: Java RMI, CORBA, Microsoft DCOM

  19. Robotic Middlewares • IPT (1996) • Object-oriented, message passing toolkit • Unmanned Ground Vehicle • Client/Server Model • MIRO (2002) • Middleware for Mobile Robots • Based on CORBA • Broker (2005) • IPC Toolkit for Multi-Robot Systems • Works on Publish/Subscribe Model

  20. CBWare Architecture • Works on Publish/Subscribe Model • Two types of Interfaces • CBQueues • CBPackets • Log server • Remote monitoring

  21. Dedicated Logging Machine • Separate Machine – Why? • Hard disk – Most sensitive part of a machine • Bumps – Common reason for hard disk crash/failure • Disk crash – Affects autonomous operations • Provision to log only sample of data to disk

  22. Sampling data for remote monitoring • Why sample data sent over wireless network? • Wireless network cannot handle all the data produced • Onboard network – 1 GB LAN Ethernet switch • Wireless network – 802.11g Wireless • Bandwidth • Ethernet LAN: 1000 Mbps • 802.11g wireless: 54 Mbps

  23. CBQueues • Interprocess communication on a single machine • Interface to read/write messages • Built using POSIX Shared memory

  24. Shared Memory Model • Area of memory shared by multiple processes. • Shared Memory area -indistinguishable to a process from unshared memory Code Code Private data Private data Shared data Shared data

  25. Why Shared Memory Model? • Fastest form of IPC available • Negligible communication overhead • Need to deliver high bandwidth sensor data in a timely manner

  26. CajunBot’s Shared Memory Model • Queues for every message type in shared memory area • Messages in every queue temporally ordered • Crucial for Interpolation support

  27. Single Writer Restriction • Only one writer for each message queue, no limit on the number of readers • Enables temporal ordering of data in distributed queues • Multiple producers for same message type • Separate queue maintained for each producer

  28. Data Fusion and Interpolation • INS • generates data at 100 Hz • Produces data at 10 ms interval • LIDAR • LIDAR generates data at 75 Hz • Scans at 13 ms interval • Most recent INS data may be up to 10 ms old when LIDAR scan is read

  29. Stabilization of sensors • In CajunBot, no stabilization of sensors • Fusing most recent data can give erroneous results • 0.5 degree inaccuracy of angular difference -> 2 feet displacement of global point from correct location • Leverage rough terrain to increase visibility • In vehicles with sensor stabilization, can fuse most recent data • Stabilization dampens rotating movements • Sensors wont experience significantly different orientations in the 10 ms period

  30. CBPackets • Distributed interprocess communication across machines • UDP Broadcast • Support for multiple readers and writers

  31. Why UDP Protocol? • Real-time applications • Deliver messages in time • Small Packet Header overhead • TCP Header = 20 bytes, UDP Header = 8 bytes • For a 20 byte message, compare: 40 bytes Vs 28 bytes Example: Case 1: Message Size = 12 bytes Efficiency using TCP = (12 / 12+20) = 0.37 = 37% Efficiency using UDP = (12/ 12+8) = 0.6 = 60% Case 2: Message Size = 1000 bytes Efficiency using TCP = (1000 / 1000+20) = 0.98 = 98% Efficiency using UDP = (1000 / 1000+8) = 0.992 = 99%

  32. Why UDP Protocol? – contd. • Broadcast/Multicast capabilities • Send messages to all processes at once • Connectionless nature • Useful for remote monitoring • No problems if wireless network fails

  33. Data Marshaling • Why data marshaling? • Distributed System – Machines with heterogeneous architectures/OS follow different byte-ordering, alignment strategies • Data converted to neutral format for transmission over the network • Examples of Data Encoding standards: XDR, NDR, CDR • XDR used by CBWare

  34. XDR Translation Decode data Encode data Receiver Sender XDR Format • Encode: Convert data from native format to XDR format • Decode: Convert data from XDR format to native format • Every message marshaled by CBWare before sending over network

  35. Monitoring Process Status • Processes on multiple machines generate status/warning/error messages • Monitoring messages on each machine • Tedious task when system scales to higher level • Cbmesg: Interface for logging and real-time monitoring of process status information

  36. Replication of Shared Memory • Combination of CBQueues and CBPackets • Subscriber replicates local shared memory • Distribute queues to multiple machines • Easy transfer of programs • Scalability of computational power

  37. CBWare Network Evaluation • CBWare evaluation metrics • End-to-End Latency (msec) • Packet Rate • Bandwidth (bytes/second) • Packet Order • Experimental setup • Dell Poweredge servers • 1 GB Ethernet LAN switch

  38. CBWare Network Evaluation Results • End-to-End Latency • Delays ranging from 0.4 msec to 6 msec depending on packet size • Transmission delay = Packet reception time – Packet sent time

  39. CBWare Network Evaluation Results – contd. • Bandwidth and Packet Rate • Packet Rate = (Bandwidth/Message Size) • Maximum rate ranging from 1200 packets/second to 90 packets/second • Packet order • Measurement based on timestamp • No out of order packets on CBWare network • No network congestion • Network Monitoring Tools used: Ethereal, tcpdump

  40. Conclusion • CBWare – Middleware for Autonomous Ground Vehicle • Publish/Subscribe Model • Distributed Information exchange - Shared Memory and UDP Broadcast • Support for fusion of sensor data • Log server • Sampling for remote real-time monitoring

  41. CBWare’s comparison with other Middlewares

  42. Future Work • Fault tolerance • Compression techniques for larger messages • Port CBWare to other Operating Systems

  43. Questions

  44. Extra slides

  45. Distributed Computing System of CajunBot • Onboard Computing system of CajunBot • Four machines • NTP • Separate networks

  46. Why XDR? • XDR format of data is same on all machines • Any machine can decode data encoded by any machine • Easily add machines with different architectures

  47. Log Control • Remote control of central log server (cb_logd) • TCP Connection • Communication using predefined ASCII Protocol • Three types of control signals • Enable logging data • Disable logging data • Change log directory on log server

  48. CBQueues Utilities • Remove corrupted shared memory: cb_qclean • Clean shared memory area • Read/Write to shared memory: cb_qtest • Testing and debugging problems with shared memory

  49. CBWare’s Publish/Subscribe Components • Publisher: cb_publisher • Handles socket operations • Constructs CBPackets • Marshal data • Broadcast on network • Central Log Server: cb_logd • Logs data received from broadcast network to disk • Runs on separate machine to prevent disk failure affecting autonomous operations • Provision to log only a sample of data to disk • Samples data at prescribed interval and broadcasts data on wireless network for remote monitoring

  50. CBWare’s Publish/Subscribe Components – contd. • Subscriber: cb_subscriber • Runs in two modes • Mode 1: Replicate “shared memory” queues across machines • Mode 2: Real-time monitoring on remote laptop • Easy transfer of programs to multiple machines • Easy scalability of computational power

More Related