Quadrangle Tracking Algorithm Software Migration

1. ECE 574 – Midterm Update Software Migration for Quadrangle Tracking AlgorithmWilliam Martin

2. A Brief Note About Ethics The work done for ECE 574 is completely separate from the work I do in my GTA and GRA responsibilities ECE 574 is different from most classes since there isn’t a regularly scheduled meeting time Thus, division of time tends to be on a per week or per month basis instead of on an hourly basis Hundreds of hours of research work already done this semester will not be applied towards ECE 574 in any way First month and a half of semester was spent finalizing and improving Matlab code Android application development TCP/IP communication integration Missile defense work

3. ECE 574 – Top Level Overview Task 1 – Research and Determine Best Software Environment (Microsoft Visual Studio C++, OpenCV) [95%] Task 2 – Establish Connections to and Control of External Hardware for New Software Environment [100%] Task 3 – Translate Software Algorithms from Matlab to Microsoft Visual Studio C++ [30%] Task 4 – Handle Miscellaneous Non-“One-to-One” Mapping of Programs [unknown] Task 5 – Perform Formal Timing Analysis of New Software Package and Identify Areas for Potential Acceleration [10%]

4. Software Migration Flowchart - Original Hello World Algorithms Hardware Interactions Orienting Camera Arduino Android Point Extraction Open TCP/IP Socket Microsoft Cam ThorLabs Cam Open Serial Port Rectification Pose Estimation Get Frame Write to Socket Write Serial Communication Final Integration Display Frame Close Serial Port Read from Socket Get and Display Successive Frames Hardware Incorporation Plotting

5. Task 1 – Research and Determine Best Software Environment Selection Criteria: Speed, popularity, resources, simplicity of hardware interactions, GUI availability, powerful IDE and debugger Research Resources Extensive web searches of forums and programming communities Interviews with Jacob D’Avy, Bobby Cho, Michael Vaughan, AnisDrira, Matthew Hollingsworth, MerekChertkow, Andrew York, Jason Lambert Languages Considered: Objective C, C#, C++, Python, FORTRAN, Visual Basic, Java Some languages had clear disadvantages (i.e. Objective C – antiquated, FORTRAN – not suitable for GUI development) While this project could have been done in many languages, C++ offered the best combination of characteristics required by the selection criteria

6. Task 1 – Research and Determine Best Software Environment IDEs/Methods Considered: Matlab C++ code generator, MSVSC++, Eclipse C++, Netbeans, Emacs, Notepad++, QT Matlab C++ generation only works with very small codes and was not suitable to this migration Chose Microsoft Visual Studio C++ One of the best IDEs One of the best debuggers Many other students in the lab have extensive knowledge of the platform Project Types Considered: Win32 Console Application, Win32 API, Windows Forms, MFC Initially made wrong choice by choosing Win32 Console Application (no GUI capabilities) Now using Win32 API. Time will tell if this was the right choice

7. Task 2 – Establish Connections to and Control of External Hardware for New Software Environment This step was identified as one of the most crucial This step was also expected to the one with the highest potential for failure Thus, higher priority was given to this step In short, Task 2 is 100% complete!

8. Task 2 – Establish Connections to and Control of External Hardware for New Software Environment Many problems were faced acquiring images MSVC++ has hundreds of project options that need to be properly set in order to work with an external library In the end, a half dozen or so of these properties were properly set and image capture succeeded (thanks to Bobby Cho)

9. Task 2 – Establish Connections to and Control of External Hardware for New Software Environment More problems were faced talking to Arduino A program that could read/write to Arduino was quickly developed, but integration with the image capture program was impossible At this point, other development environments were considered In the end, new libraries were written and the decision was made to continue using MSVC++

10. Task 2 – Establish Connections to and Control of External Hardware for New Software Environment

11. Task 2 – Establish Connections to and Control of External Hardware for New Software Environment Establishing connection with Android posed the fewest problems Original attempts to modify Michael Juggan’s code failed (code written in C#) Extensive web searches for TCP/IP communications in MSVC++ initially returned few results Eventually, an generic library was found, rewritten and implemented fairly quickly While the program and the android device can both read and write to each other, the complete communication routines will need to be implemented as more of the algorithms are translated

12. Task 3 – Translate Software Algorithms from Matlab to Microsoft Visual Studio C++ Once all hardware hurdles had been passed, software migration could begin Due to the varied differences between Matlab and C++, this is a very slow, labor intensive process Many times, the translation is simply changing of syntax and data types While many logical functions are similar between the two environments, some Matlab functions have no C++ analogs and need to be written from scratch So far, point extraction and orientation algorithms have been successfully migrated Generally, this task is expected to go smoothly

13. Matlab to Microsoft Visual Studio C++ Software Migration Flowchart - Updated Hello World ✓ Algorithms Hardware Interactions ✓ ✓ Orienting Camera Arduino Android Point Extraction ✓ ✓ ✓ Open TCP/IP Socket Microsoft Cam ThorLabs Cam Open Serial Port Rectification Pose Estimation ✓ ✓ Get Frame ✓ Write to Socket Write Serial Communication Final Integration ✓ ✓ Display Frame 30% ✓ Close Serial Port Read from Socket ✓ ✓ Get and Display Successive Frames ✓ GUI Plotting Hardware Incorporation Full Arduino Communication Code Rewrite Full Android Communication Code Rewrite 25% 20% 15%

14. Task 2 – Establish Connections to and Control of External Hardware for New Software Environment

15. Task 4 – Handle Miscellaneous Non-“One-to-One” Mapping of Programs Some non-linear translation must be done Creating circles example Sorting algorithm example Matrix manipulations Otherwise, Task 4 is where the size of the project can grow/shrink to appropriate size for ECE 574…

16. Task 4 – Handle Miscellaneous Non-“One-to-One” Mapping of Programs Plotting Generating a 3D model like we did in Matlab may prove to be an enormous task Since the Android will be the main controller, this task may be redundant GUI Development of a GUI is something that has no Matlab analog. The complexity of the GUI is a place where we’ll have room to grow/shrink the project as necessary New, Faster Algorithms Would like to explore using the Hough transform for circle (and therefore point) extraction Other opportunities to increase speed not yet encountered

17. Task 5 – Perform Formal Timing Analysis of New Software Package and Identify Areas for Potential Acceleration Simple timing calculations were done for image differencing Initial timing comparisons in the new software environment are encouraging At this stage of development, an average frame can be processed in ~300ms as opposed to ~1800ms in Matlab (6x faster) Extensive timing analysis will be done at the end of the semester and will determine the next steps in the project

18. Future Work (In Rough Chronological Order) Create/Update Website Continue development of more intuitive separation of code into object-oriented classes Translate pose estimation algorithm Translate image rectification algorithm Incorporate full communication protocol with Android device Expand error catching and handling capabilities to ensure program stability Development of GUI Pursue other work as necessary for scope of project (Graphing? Hough transform? GUI? Others?) Complete Task 5

19. Conclusions/Summary So far, it seems that the choice of language and IDE have been good ones Despite a lack of experience with MSVC++, many of the initial hurdles have been passed By completing the hardest tasks first, we are consistently increasing the chances of project success as time goes on Since the work ahead is mostly mundane, progress on the project should accelerate Initial results are promising and point towards a significant improvement in frame rate at the end of the project

20. Questions