3D Environmental Mapping and Imaging for AUVSI RoboBoat

1. 3D Environmental Mapping and Imaging for AUVSI RoboBoat David Bumpus, Dan Kubik, & Juan Vazquez Advisor: Dr. José Sánchez Customer: Mr. Nick Schmidt Department of Electrical & Computer Engineering November 19, 2015

2. Problem Background: Significance • AUVSI – Association for Unmanned Vehicle System Int’l • International RoboBoat Competition • Bradley has attended since 2012 Fig. 1. Bradley 2013 RoboBoat [1]

3. Problem Background: Significance • Boat undergoes challenges Fig. 3. 2014 RoboBoat course [2] Fig. 2. RoboBoat navigating through course [1]

4. Problem Background: Motivation Fig. 4. Boat navigation [3] Fig. 5. Boat struggling [3]

5. Problem Background: Objective • Return: • Distance measurements • Camera image • Image with distance information • Location of nearest object

6. David Bumpus: Work Accomplished • Research and develop registration method • Simulate registration method in MATLAB

7. David Bumpus: Work Accomplished

8. Work Accomplished: Research Registration [4] • Read papers on Lidar and image registration • Feature Detection vs. Mutual Information • Mutual information is computationally expensive • Feature Detection is dependent on presence of sufficient number and distribution of features Lidar Point Cloud Lidar Intensity Fig. 6. Mutual Information Principle [4] 2D Image

9. Work Accomplished: Research Registration [5] • Scale Invariant Feature Transform (SIFT) • Speeded-Up Robust Features (SURF) • Principal Components Analysis SIFT (PCA-SIFT) Table I- Performance Comparison of 3 Algorithms [5]

10. David Bumpus: Work Accomplished

11. Work Accomplished: MATLAB Simulation Fig. 7. MATLAB SIFT Algorithm Implementation [6]

12. Work Accomplished: MATLAB Simulation • Vectors represent keypoint descriptors • Each descriptor describes: • Location of keypoint • Magnitude of gradient • Direction of gradient Fig. 7. MATLAB SIFT Algorithm Implementation [6]

13. Work Accomplished: MATLAB Simulation Fig. 8. Matched features for two very different images [6]

14. Work Accomplished: MATLAB Simulation • Able to match features from very different environments • Some error present, but attribute to relative object shifts • Ball has moved in relation to chalkboard and camera angle has changed Fig. 8. Matched features for two very different images [6]

15. Work Accomplished: MATLAB Simulation Fig. 9 Matched features for two similar images [6]

16. David Bumpus: Work Accomplished

17. David Bumpus: Future Work • Implement registration on embedded device for single frame • Delaunay Triangulation • SIFT Feature Matching • RANSAC Application • Produce depth overlay • Implement registration for live video

18. Juan Vazquez: Work Accomplished • OS Selection & Installation • Logitech C500 & Odroid-XU4 Interface • VLP-16 & Odroid-XU4 Interface 70%

19. Juan Vazquez: Work Accomplished 70%

20. Work Accomplished: OS Installation • Ubuntu Server • Memory usage • Software availability • Configurability Fig. 10. Ubuntu Server [7]

21. Work Accomplished: OS Installation • Operating system booting • MicroSD • EMMC (Embedded MultiMediaCard) • Establishing network connection • Ethernet connection bridge Fig. 12. EMMC [9] Fig. 11. MicroSD [8]

22. Juan Vazquez: Work Accomplished 70%

23. Work Accomplished: Camera Interface • Logitech C500 Linux driver • OpenCV • Functions library • Video capture • Image registration Fig. 13. OpenCV [10]

24. Work Accomplished: Camera Interface • Image capture program • Live video read • Continuously capturing/overwriting • Stores in PPM (Portable Pixmap Format) Fig. 14. Logitech C500 [11]

25. Juan Vazquez: Work Accomplished 70%

26. Work Accomplished: VLP-16 Interface • Libpcap • Library for packet capture • Packet Detection • UDP (User Datagram Protocol) • Packet Storage • Text file Fig. 15. VLP-16 [12]

27. Juan Vazquez: Future Work • VLP-16 & Odroid-XU4 Interface • Packet Formatting • Accuracy Testing 70%

28. Dan Kubik: Work Accomplished • Simulation of lidar (MATLAB) • Implementation of lidar reading functionality (C++) 80%

29. Dan Kubik: Work Accomplished 80%

30. Work Accomplished: Simulation of lidar • VeloView Software Fig. 17. VeloView view of box Fig. 16. VeloView full view of room with box

31. Work Accomplished: Simulation of lidar • Keys to successful simulation: • Visualizing how data will be received • Stream of bytes vs. full packet • Understanding data structure • Packets, blocks, data points • Conversion from spherical to rectangular coordinates

32. Work Accomplished: Simulation of lidar • MATLAB Fig. 19. MATLAB view of box Fig. 18. MATLAB full view of room with box

33. Dan Kubik: Work Accomplished 80%

34. Work Accomplished: Lidar Implementation • Goal: Implement for Odroid XU4 • C++ and Object Oriented Programming • Creation of classes: • Storage classes (DataPacket, DataBlock, HalfDataBlock, sphDataPoint) • Interface classes (MasterBlock, recDataPoint)

35. Dan Kubik: Future Work • Implementation of lidar reading functionality • Installing code to Odroid XU4 • Testing 80%

36. Conclusion • David • Registration algorithm development • Juan • OS installation & interfacing • Dan • Simulation & implementation of lidar

37. Conclusion • Progress and Future work: • On schedule • Potential problems with registration • Difficulty obtaining key points from lidar data

38. 3D Environmental Mapping and Imaging for AUVSI RoboBoat David Bumpus, Dan Kubik, & Juan Vazquez Advisor: Dr. José Sánchez Customer: Mr. Nick Schmidt Department of Electrical & Computer Engineering November 19, 2015

39. Additional Slides

40. Testing of VLP-16 Puck™ • Install Veloview Software • Test acquisition of data to Veloview and verify distances • Familiarize self with Veloview data manipulation Fig. 20. Data acquisition to Veloview using VLP-16 Puck

41. Testing of Logitech C500 Webcam • Install Logitech software • Capture images and Video Fig. 21. Color image from Logitech C500

42. Registration Flowchart • Filter point cloud • Interpolate point cloud • Create depth maps • Feature detection • Outlier elimination • Feature matching Fig. 22. Registration method flowchart [13]

43. EMMC vs. microSD Card Fig. 23. Write/Read Speed Comparison [14]

44. UDP Detection • Current Packet Sniffer Detection • TCP (Transmission Control Protocol) • UDP (User Datagram Protocol) • ICMP (Internet Control Message Protocol) • IGMP (Internet Group Management Protocol) • Others (Detects all Ethernet Frames) Fig. 23. VLP-16 UDP Detection [15]

45. Ubuntu Desktop vs. Server Fig. 24. Ubuntu Desktop Requirements [16] Fig. 25. Ubuntu Server Requirements [16]

46. Creating a Bootable MicroSD Card • MiniTool Partition Wizard • Partition creation and wiping • 7zip • Extraction of IMG.XZ file • Win32 Disk Imager • Writing the unpacked IMG file to Micro SD card • Process can also used for flashing eMMC

47. OpenCV System Requirements [17] • Operating System: Windows, Linux or Mac • Memory (RAM): 256 MB • Hard Disk Space: 50 MB • Processor: 900 MHz

48. JPG, PNG, & PPM Comparisons [18] [19] • JPG (Joint Photographic Experts Group) • Supports 24-bit color RGB • Compress image data by reducing sections of images to blocks or pixels • Compression is permanent • PNG (Portable Network Graphics) • Supports transparency, 8-bit color, and 24-bit color RGB • Largest of three compared formats • PPM (Portable Pixmap Utilities) • Lowest common denominator color image file format • Includes basic color • Easy to format to process

49. Data Diagram: Velodyne User’s Manual

50. Storage Classes • DataPacket: Contains 12 DataBlocks and a time stamp • DataBlock: contains 2 HalfDataBlocks • HalfDataBlock: contains 16 DataPoints and an azimuth • DataPoint: contains one distance, reflectivity, and alt. ang.