Computer Science Capstone Team Trimaxion

1. Computer Science Capstone Team Trimaxion Presented by Team Trimaxion Jon Price, Eric Enger, Jason Farrell Tobias Knutsson, John-Oskar Ahlstrom, Oscar Almgren, Robin Malmros

2. Introduction • System Design • The design of the system was initially drafted very early • We wanted to keep things simple but also allow for complexity where needed • Components should be standalone and require only knowledge of what they are receiving/sending • Limit redundant code • Division of components • Video Server • GUI • Main Server (Image Processing, Path finding) • Brain

3. Introduction (cont) • Goals • The robot should allow for manual control within given values being sent from the GUI • A live video stream should be displayed in the GUI and be clickable • When the video is clicked the destination should be determined and the path automatically calculated and followed avoiding obstacles when necessary • Upon completion the robot should be able to acquire a target and fire a projectile at the target

4. Introduction (cont) • Methodology • Components should be standalone and be able to work alone • Test a completed component with a test application • Perform weekly tests to ensure frozen code works with new code • KISS – Keep it Simple Stupid – Nothing Unnecessary, add extra when product is complete • Maintain constant communication and provide continual progress updates

5. Introduction (cont) • Areas of Technical Expertise • Robot being Completely autonomous when finding a path • Light based movement sensor with pin wheel • Projectile launching system

6. General Overview

7. Development of Trimaxion KISS – Keep It Simple Stupid

8. Development Problems • In General • A wide variety of programming knowledge and experience was exhibited by the members of our group • Main Server • The Main Server took longer to complete then usual • Communication with Sweden was at times lost for unknown reasons • Robot • Sliding Treads • Battery Power • Jon will talk more about this later on

9. Tools • Google Code • Download Features • Shared SVN • Each Component was given its own folder under ‘trunk’ • Problem – this was not the best idea as certain projects needed to be able to reference certain classes in other project • Example: Command.java is used in both GUI and Main Server • BaseCamp • Managed Milestones and Global ToDo list • Eclipse w/ SubEclipse • Netbeans

10. Tools (cont) • Java • Java Media Framework • Java Advanced Imaging • Lego Mindstorms • Runestone • Team Communication • Deliverables hosting

11. Video Server Presented by Jason Farrell

12. Research • Real Time Transfer Protocol • UDP based to allow efficient data transfer with permissable data loss • Stream a media location that can be read by a client via the RTP protocol implementation in JMF • Java Media Framework (JMF) • Allow access to media capture device (microphone, webcam) • Use data source to display video or produce audio • Use RTP Session Manager to broadcast data to client • Use of Player and Processor objects to manage and display stream data

13. Development • First goal was to be able to display the output of the data source in a GUI • Accomplished using the Data Source class in conjunction with a player object designed to interpret that data source • Next be able to transfer the data source over the network • To test this we used JMStudio which came with the framework and was developed by Sun Microsystems. • To assist with the development of the transmitter we used the VideoTransmit class from the Sun Developer network. • We specified the destination via RTP by specifying an RTP address of the format: rpt://ip.address:port/track

14. Testing • The primary means of testing was the use of JMStudio, which we knew to be a working program, thus to make it easier to determine which component was failing. • First segment, testing the transmission by opening an RTP Session from JMStudio • Second Segment, testing the reception by transmitting from JMStudio • Final Segment, testing using both the GUI and VideoServer for Reception and Transmission respectively.

15. Graphical Front End Presented by Eric Enger and Jason Farrell

16. Research • What would the GUI look like? • Manual movement commands for rotation and straight line travel • Streaming web cam image, clickable to allow point selection (JMF) • Communications protocol with Main Server • TCP with Java Sockets • What to send between? • Command object • Image, Point, Command • Object I/O Streams

17. Development • Design • Needed to support JButton and JTextField for manual robot control parameter entry • Communicate with Main Server using Object I/O Streams • Leverage Java for all components • Limit number of mouse clicks to send info to server • Command class houses instances of ImageIcon, Point, and Command • Command is a char field that dictates what type of command is being sent • Use of JMF to read the RTP Data Stream coming from Video Server • JMF also used to “capture” the current image in data stream

18. Development (cont) • Command.java Implementation • Supports a char field that is accessed by Main Server to determine command type • M – move • R – rotate • P – Path Find • Image is the captured image from the web cam stream. Stored as an ImageIcon but accessed as an Image using encapsulation • ImageIcon is inherently serializable • Point – Instance of point generated by mouseClick event associated with VisualComponent of JMF Player

19. Testing • We used the finished GUI with a test server involving stub code from Main Server. • Goal: send the image and point across the Grand Valley Network • Results: Transmitting Computer must be plugged into the wall • Have finished GUI attempt to connect and send image to Test Server located in Sweden • Goal: Display the image from the webcam in Sweden • Result: All computers involved must be connected via a wall connection with no firewall

20. Main Server – Image Processing Presented by Tobias Knutsson and John-Oskar Ahlstrom

21. Research • http://google.com/search?q=java+image+processing • Java Advanced Imaging (JAI) • Platform independent (Java Interface) • Fast (C Backbone)

22. Development • Needed abilities: • Locate obstacles • Locate robot position and relative angle • Remove robot from the obstacle map • Determine map scale (using the robot as a reference) • Development approach: Find solutions that work in different environments (lighting conditions, surfaces etc.) • The web is an invaluable resource for examples. We do not want to reinvent the wheel

23. Unprocessed Image in the Visor

24. Processed Image from Visor

25. Testing • Challenge: Designing the algorithm using static images and keeping it flexible • Solution: Try to create realistic scenario-photos, but change lighting, obstacles surfaces • Scaling images gives better performace, but also gives you alot more factors to keep track of

26. Main Server - Pathfinding Presented by Robin Malmros, Oscar Almgren

27. Research • Discussion led to Guiding Star (A*) • What is Guiding Star? • Google revealed excellent sources

28. Development • Coding in Java using Eclipse • Optimized the path with the removal of intermediary points • Integrated the path finding into Astrolabe

29. Testing • Testing of arbitrary boolean map • Testing of map received from Command • Testing with full system integration

30. Robot Presented by Jon Price Assisted by Eric Enger, Tobias Knutsson and John-Oskar Ahlstrom

31. Deciding on the Build • Robot must be able to turn without deviating from its current location. • Decided to use a tank design. • Initial navigation would use the TiminingNavigator class developed in Lejos. • Communication would be achieved using the RCXPort provided by Lejos.

32. Robot Design Phase The final design was a modification made by the team in Sweden. This design placed the IR sensor towards the ceiling and adding a well designed bumper sensor to the front. Our initial robot design was a very basic tank design that was found using Google. The design was modified slightly to place the IR sensor in the back of the robot.

33. The TimingNavigator Class • The TimingNavigtor class was written by the developers of Lejos. • It provides (somewhat) accurate movement based on how long it takes for two functions to be completed • How long does it take to travel 1 meter. • How long does it take to rotate 360 degrees.

34. TimingNavigator, cont. • The measurements obtained depend on the surface used and the voltage of the batteries. So, the measurements would actually change over time. • It was determined a better solution was needed. • Tobias saves the day when he discovers the “poor-mans” rotation sensor.

35. Rotation Sensor • The encoder wheel is attached to a large gear. • The light sensor is used to read white and black values on the wheel. • A count is kept during movement.

36. Rotation Sensor, cont. • Provides accurate movement regardless of battery voltage or motor speed. • Still requires measurements to be taken, but they should hold true for that situation. • Measurements required: • Count to travel 1 centimeter • Count to rotate 1 degree

37. Rotation Sensor, cont. • The rotation sensor did introduce some new headaches. • Find the largest number of sector’s on the wheel and still be accurate. • Find the average value the light sensor “sees” for black and white areas. • Use trial and error to calculate 1 centimeter and 1 degree. • What is the accuracy obtained • Smallest movement allowed • Smallest turn allowed

38. The LightNavigator class • Class developed to accurately move using the new rotation sensor. • Once the class is instantiated, all you need to do is tell it to travel ‘x’ centimeters or rotate ‘x’ degrees. • Drawback: Does not maintain current x and y position of the robot.

39. Communication with RCXPort • Decided to use the RCXPort already developed for Lejos. • Slow speeds, but provides reliable connection and ensures all data is transferred. • Easy to work with because it uses Java’s Data IO methods. • Develop RCXProtocol class to bridge the server <----> RCX communication.

40. RCXProtocol Class • Simple class that creates the RCXPort and retrieves the input and output streams. • Provides methods for communicating with the robot: • Travel ‘x’ centimeters • Rotate ‘x’ degrees • Sends the commands and blocks until a response from the RCX is received.

41. Continuous Testing • During the design phase there was continuous testing of the robot’s “brain”. • Movement always seemed to be hit or miss when it came to accuracy. • What was the cause of the inconsistency? • X and Y coordinates, along with robot’s rotation were inaccurate. • Once removed/not used we were able to provide more accurate movement.

42. Inconsistent Communication with RCX • There are still inconsistencies with the way the RCX communicates with the server. • Commands received, executed, no acknowledgement • Commands never received. • One solution was to add some minor delays after commands are sent and executed.

43. Final Words • Bumper Sensor • Stops all movement and plays a series of tones. • No recovery at this point • Possibilities? • Lost Communication • Java IO Blocks, so it was hard to “timeout” the IO streams on the RCX. • RCXPort provides reliable data transfer, so no packets will be lost. • Possibilities?

44. References “Poor Mans” Rotation Sensor: http://www.restena.lu/convict/Jeunes/Divers/Poor_rot.htm Lejos Download and API: http://lejos.sourceforge.net/ Initial Tankbot Design: http://www-education.rec.ri.cmu.edu/robo_preview/content/mult/slide/tankbot.htm

45. Testing & Integration Presented by Team Trimaxion Cause you don’t want end up living in a van down by the river

46. Introduction • Testing of VideoServer and GUI across the Atlantic to determine connectivity requirements • Localized testing to determine components interaction locally • Full testing involving connections between America and Sweden

47. VideoServer to GUI Testing • After completion of localized testing we set out to “meet the Swedes” and get them on cam • We acquired a firewall free connection from Carl Strebal and proceeded to assist the Swedes with getting the code working on their end • We discovered that for optimum results both sides needed to be connected to the wall • Also during this testing process we decided on a consistent port access protocol • Any connection from America to Sweden uses port 22224 • Any connection from Sweden to America uses port 22222

48. Localized Testing • Problems: Finding a computer was the hardest portion • We tried a number of machines before settling on Eric’s machine to host Astrolabe (Main Server) • Over calculation of scaled values – robot moving too far • Required network testing to ensure proper communication. • Jason: Video Server /GUI • Eric: GUI / Main Server (Astrolabe) • Integration: Fusing atypical code • Environmental variable problem • .dll files and pathing

49. Conclusions Derka Derka

50. What we would do over? • Communication Redesign • Blue Tooth • Multicast Video Server • Redesign of movement calculation for robot • Projectile launching device