Revised Proposal for William Martin’s Graduate Research Area

1. Revised Proposal for William Martin’s Graduate Research Area

2. Outline For Today’s Presentation: Reminder of Initial Idea Quadcopter/Kinect Drawbacks to the original plan Refined Idea Main addition – Tablet-based controller New project philosophy Project Architecture Recruiting Project Team Members Why we should recruit more people? Prospective Members Conclusions

3. A: Reminder of Initial Idea • Quadcopter/Kinect Photo credit: Greg Bradley via Radaris.com

4. A: Reminder of Initial Idea • Quadcopter/Kinect • Use pre-existing UTK quadcopter project as starting point for graduate research • Mount a Microsoft Kinect sensor onto quadcopter • Mount local computer onto quadcopter to make use of Kinect data • Refine automatic stabilization algorithms already loaded onto quadcopter • Use these data to perform image recognition to allow the unit to autonomously explore 3D space • Integrate object recognition algorithms (where the objects are humans) to allow quadcopter to be used in search and rescue operations

5. A: Reminder of Initial Idea • Drawbacks to Original Plan • Current plan does not include an ability to see live video data from quadcopter’s perspective • This plan is not particularly novel, since similar projects have been completed at other universities • Locally mounted computing hardware will add weight to an already heavy and underpowered flying device • Specifying “search and rescue” as the main purpose of the device is unnecessarily restrictive

6. Outline For Today’s Presentation: Reminder of Initial Idea Quadcopter/Kinect Drawbacks to the original plan Refined Idea Main addition – Tablet-based controller New project philosophy Project Architecture Recruiting Project Team Members Why we should recruit more people? Prospective Members Conclusions

7. B: Refined Idea • When I attempted to pilot the existing quadcopter, I found the controller very difficult to use and unintuitive • A tablet, such as an iPad or Android-based device would allow for intuitive, accelerometer and/or GUI-based controls • Main Idea – Tablet Based Controller

8. B: Refined Idea • Advantages To This Approach • Would allow for live RGB video feed of quadcopter’s view • Would allow for live SLAM-based 3D rendering of environment • Processing could be moved off of local computer and onto tablet • A new user could easily pilot the quadcopter with very little practice • Main Idea – Tablet Based Controller

9. Main Idea – Tablet Based Controller B: Refined Idea Live RGB video feed SLAM-based 3D map HUD with vital information Touch-based analog controller 1 Touch-based analog controller 2 Status display Mode selection Battery Information Menu 7 Mock-up of Tablet display 3 2 1 4 5 6 9 8 (Images are of MIT’s STATA Center)

10. B: Refined Idea • Don’t Define the Purpose Up-Front! • No need to pigeonhole ourselves into a specific application • Instead, create a robust, multi-functional, intuitive flying/imaging/3D-model creating-device • Don’t Define the User Up-Front! • While manual flight controls will be available, the burden of actually operating the vehicle will be on the software, not the user • By navigating intuitive menus on the tablet, a completely inexperienced user can dispatch the vehicle on a mission suited to their specific purposes within hours or even minutes • New Project Philosophy

11. B: Refined Idea • Equip the vehicle with a suite of multipurpose, commonly-needed functions: • Scan/image 3D object • Track human/moving object in frame • Point-to-point, GPS-based navigation (with obstacle avoidance) • Create RGBD model of a rectangular area defined by GPS center point • Light cargo transportation • Explore indoor space • Search space for specific objects (humans, moving objects, etc.) • Allow user to capture image of GUI display at any point in flight, a video of the entire flight, and partial or full 3D models created during flight • New Project Philosophy

12. B: Refined Idea • New Project Philosophy • Maximize Autonomy: • Automatically retrace path and return to starting point before battery failure • Stabilization of pitch, yaw, roll, and directional vector all done by onboard hardware and software • Takeoff and landing procedures executed on-board • Automatic height maintenance determined by current task • Once user defines the task to be completed, minimal to zero interaction with controller will be necessary

13. Outline For Today’s Presentation: Reminder of Initial Idea Quadcopter/Kinect Drawbacks to the original plan Refined Idea Main addition – Tablet-based controller New project philosophy Project Architecture Recruiting Project Team Members Why we should recruit more people? Prospective Members Conclusions

14. C: Project Architecture Hardware Software (Needed on Quadcopter and Tablet) • Quadcopter • Encoders • Servos/motors • Underlying structure • New Components • z-axis range finder? • GPS unit • Battery • New servos/motors? • Kinect (deconstructed) • Tablet (iPad or Android Device?) • Communications • USB to wifi forwarder? • On-board, Linux-based wifi server? • Stabilization (may be hardware issue as well) • Pitch/Roll • Directional vector (yaw) • Position • Height • Communications • Receiving position/direction info • Receiving Kinect data • Transmitting instructions • GUI • Display, menus • Algorithms • Function suite • Hardware/software/communications/GUI integrations

15. C: Project Architecture • The main functionality of the tablet and its interaction with the Kinect data can be developed independent from the quadcopter • Much of the stabilization/automation of the quadcopter can be developed independent from the Kinect and the tablet

16. Outline For Today’s Presentation: Reminder of Initial Idea Quadcopter/Kinect Drawbacks to the original plan Refined Idea Main addition – Tablet-based controller New project philosophy Project Architecture Recruiting Project Team Members Why we should recruit more people? Prospective Members Conclusions

17. D: Recruiting Project Team Members • Why should we recruit more people? • The project I’ve outlined today is probably bigger than a single master’s thesis • Why not try to make this the flagship project for UTK’s IRIS department? • The more competent, enthusiastic people that we can get working on this project, the bigger and better it will be • Given my career goals, its important to me to get some project management experience • Even with a number of additional team members, there is still plenty of work available for me to carve a thesis out of • I’d be happy and comfortable doing any portion of this project as my master’s thesis

18. D: Recruiting Project Team Members • Prospective Members • Ali Ghezawi • Rising senior EE • 4.0 GPA • A singular genius • Has already completed senior design, but could probably be recruited for a special topics course • Talents – Math, modeling, Matlab, Labview, Microelectronics • Jonathan Edwards • Rising senior EE • President – Eta Kappa Nu • Extremely competent, studious, and hard-working • Needs to take ECE 400 • Talents – Image processing/pattern rec, software

19. D: Recruiting Project Team Members • Prospective Members • I’ve already approached Jonathan Edwards and Ali Ghezawi about this project • Both voiced interest, but recruitment by the IRIS lab may be needed to get a firm commitment • Other seniors in need of senior design projects would be more than welcome • Graduate students who would like to be involved in the project would be welcomed as well • I plan on contacting Dr. Crilly as a communications mentor • I also plan on talking to Drs. Birdwell, Roberts, Qi, and Gregor about getting involved project as well

20. Outline For Today’s Presentation: Reminder of Initial Idea Quadcopter/Kinect Drawbacks to the original plan Refined Idea Main addition – Tablet-based controller New project philosophy Project Architecture Recruiting Project Team Members Why we should recruit more people? Prospective Members Conclusions

21. E: Conclusions Main Challenges/Questions Presented by This New Approach • How will we get the Kinect data to the tablet? • How will we get the results of the navigational algorithms from the tablet to the quadcopter? • Are tablets capable of processing Kinect data and SLAM algorithms in real time, or will we need to tether the tablet to another processor? • What happens if the quadcopter cannot establish communication with the tablet? • If the communication link is lost, will the computations necessary to continue the mission require a microprocessor, or will a microcontroller suffice?

22. E: Conclusions Take-Away Points • Single Sentence Summary: “By adding GPS, a Microsoft Kinect, autonomous stabilization software/hardware, and a server to the existing quadcopter, we will enable a tablet (i.e. iPad2) to execute navigational and imaging algorithms that can render live, RGB and 3D displays of the environment that allow for uniquely intuitive controls.” • If the system described above is achieved, it will be unique, intuitive, multifunctional, customizable, bleeding-edge, and a major contribution to IRIS Lab’s prestige. • The more bright minds we bring to the project, the better the final product. • Since project management experience is something I want to gain as a graduate student, I’d like to be the development team lead (under Dr. Abidi’s guidance and tutelage). • No matter the size of the team, there is plenty of work here for me to make a thesis out of.

23. Questions