Project on Unmanned Aircraft in the NAS Mid-semester Review Panel Meeting

1. Project on Unmanned Aircraft in the NAS Mid-semester Review Panel Meeting

2. Expert Review Panel • Tom Curtin, AUVSI & ONR • Jim Geibel, GAO • Tom Henricks, AW&ST • Ramon Lopez, Aurora Flight Sciences • Molly Macauley, Resources for the Future • James Sizemore, FAA • John Walker, RTCA SC-203 • Dyke Weatherington, DoD/OSD

3. Integration of Unmanned Aircraft into the National Airspace System A Project Course by Carnegie Mellon University Dept. of Engineering and Public Policy Dept. of Social and Decision Sciences March 5, 2007

4. Purpose of CMU Project Courses in Technology and Policy • Analyze a “real world” policy problem involving technology • Combine diverse information and analytic frameworks to derive policy insights • Learning objectives: • Problem decomposition, structuring and formulation • Interdisciplinary problem solving • Communication • Teamwork

5. Examples of past project courses

6. Contributors to our UAS project • 20 undergraduates majoring in: • Engineering • Social Science • Business Administration • 3 Ph.D. student managers • 3 faculty advisors • Expert panel review

7. Role of Expert Review Panel • At mid-semester: • Review project team’s progress and plans • Suggest changes in project direction • Suggest better ways to obtain data and conduct analysis • At semester’s end: • Provide feedback on draft of final report

8. Background for this Project • Increasing demand for UASs • Military (many current uses) • Civilian (many potential uses) • Federal Aviation Administration (FAA) is developing a roadmap for integrating UASs into the NAS • A few of the issues to be addressed: • Safety and reliability • Public acceptability • Market viability

9. Project Strategy • Pursue a set of analyses that will advance understanding of the UAS-NAS integration problem, making best use of the skills of project members, while being realistic about the project's resource and time constraints

10. Analysis Teams • Technology and Risk • What are the regulatory implications of different approaches to “equivalent level of safety?” • Public Awareness and Perception • Are risks of UAS of greater public concern than risks of manned aircraft? • Economics • How cost-effective are UASs compared to alternative means of providing the same service? • Governance • How can the current system for deliberation and decision-making on UAS access be improved?

11. Technology and Risk Team Members Samiah Akhtar Jon Cornell Nicole Hayward Will Kim Nick Misek Doug Robl Manager Leonardo Reyes-Gonzalez

12. Outline • Issues that we plan to address • Our approach • Deliverables

13. Issues to be addressed 1. What are the implications of FAA’s “equivalent level of safety” (ELOS) goal for target UAS failure rates? 2. What are current UAS failure rates and how much can they be reduced by safety technologies such as transponders, see and avoid capacity, ADS-B transceivers, AN hardware, etc? 3. How can the FAA/industry measure and track the safety performance of unmanned aircraft?

14. 1. Implications of Equivalent Level of Safety • ELOS: FAA’s goal is that risks to people in the air and on the ground from unmanned aircraft must be no greater than those from manned aircraft. • Practical Interpretation: “Equal” risk means equal number of catastrophic events nationwide. Why? • Low probability events causing large numbers of casualties have been most important drivers of policy • e.g. 1956 Airplane collision over Grand Canyon

15. 1. Implications of Equivalent Level of Safety UA catastrophic risk model Catastrophes caused by midair collision Catastrophes caused by UAS malfunction Annual UA catastrophes + = Baseline annual manned catastrophes State and control variables

16. UA catastrophic risk model Midair collision catastrophe rate Wingspan & mass # of UAs in airspace # of other vehicles in airspace Population flying Population density on ground ELOS constraint: Catastrophes per year for manned aircraft Malfunction catastrophe rate Mass # of UAs in airspace Reliability (Failures/Hr*) Population density on ground *Office of the Secretary of Defense. Unmanned Air Vehicle Reliability Study. Feb 2003.

17. 2. Technologies to Mitigate Risk • How much can a given technology reduce risk per flight hour below the status quo? • Transponders, see and avoid capacity, ADS-B, AN hardware, flight-real operator consoles. http://www.competitionaircraft.com/ultralight-aircraft-parts.html http://www.americanchampionsales.com/Assets/Images/ACA%20options/MVC-834F_JPG.jpg

18. 2. Technologies to Mitigate Risk • Identify main causes of failure gap between UAS and manned aircraft (e.g., human factors, equipment failure) • Assess potential of given technology to reduce particular failure modes from Weatherington 2006

19. 3. Tracking UAS safety incidents • Nuclear power industry vastly improved safety culture after TMI by sharing and analyzing safety information • For the UAS industry, what kinds of information collection, sharing, and analysis is feasible? • Accessible to FAA? to the public? • Voluntary or regulatory mandate?

20. Intended Deliverables • Analysis of how different UA flight variables (e.g. numbers, size, failure rates, population flown over) can be varied to achieve the ELOS goal • Description of the latest UAS technologies to mitigate risk, and estimates of their effectiveness • A discussion of alternative safety incident reporting systems

21. Review Panel Input • How significant is catastrophe rate as the ultimate driver of aviation safety performance? • Which safety technologies are most promising for large UAs? Small UAs? • How are accident and near-accident data currently collected for UASs?

22. Public Awareness and Perception Team Members Darian Ghorbi Jenny Kim Mark Peterson Laura Seitz Patrick Snyder Manager Pete Tengtrakul

23. Statement of Purpose • Add the element of public perception to the UAS-NAS discussions • Motivation: the fact that there has never been a formal presentation of public perception on the topic • Findings: useful for the creation of regulations

24. Objectives • Compare public perceptions of the risks of manned and unmanned aircraft • Find demographic groups with certain risk/benefit patterns • Research implications of opinions regarding UASs • Create and administer survey to aid in completing objectives

25. Hypotheses • Autonomous vs. Remotely Piloted - Autonomous perceived as more risky • Ground vs. Air - UASs perceived as more risky in air • Prior Knowledge vs. Risk Perception - Prior knowledge, associate less risk • Benefit vs. Risk Perception - Higher benefit, lower risk • Education vs. Risk Perception - Technical education, associate less risk • Age vs. Risk Perception - Older demographics more cautious • Frequency of Flight - Arguments for both sides

26. Layout of Survey • First Page • Provide information about UASs • Autonomous • Remotely Operated • Gauge previous knowledge • Source

27. Layout of Survey • Application • Traffic Monitoring • Pipeline Monitoring • Disaster Relief • Border Patrol • Agriculture Monitoring • Questions • Quick Response • Benefit • Stakeholder • Public • Risk • Ground • Air • 7-Point Scale • 1 - Much Less • 4 - Same • 7 - Much More Picture of UAS application • Description of UAS • Physical Information • Current Application

28. Layout of Survey • Last Page • Demographics • Gender • Age • Education • Frequency of Flight • Voting (identify opinions of those that are politically engaged) • Pilot (may have drastically different opinions)

29. Executing Surveys • Method • In person • Potential Locations • Pittsburgh Shopping Malls • Incentives • Raffle, Carnegie Mellon trinkets, $2 Bills • Quantity Desired • ~300 surveys needed for adequate statistical power to determine the different risks and benefits

30. Sample Results "Perceived Risk of UASs used in Border Patrol application, by age demographic” **Not real data, created to illustrate analysis

31. Sample Results "Perceived Risks between Remotely Operated and Autonomous UAS applications" **Not real data, created to illustrate analysis

32. Review Panel Input • Which relationships are of most interest? • Any attributes of subjects or UASs overlooked? • How will this information be useful in making decisions? • Methods of assessment other than survey?

33. Economic Perspective Team Members Reiko Baugham Brad Brown Feng Deng Nathan Diorio-Toth Victoria Morton Manager Ryan Kurlinski

34. Outline • Purpose Statement • Goals • UAS Applications • Preliminary Cost Model • Problems/Issues • The Next Step • Questions

35. Purpose Statement • Assessing the market viability of specific UAS applications using an analysis of relative cost effectiveness • Analyzing how different regulatory measures could affect costs and therefore the viability of a UAS market for a particular application

36. Goals • Develop UAS cost model • Capital vs. Operating costs • Varying UAS sizes • Compare with both ground based and aerial alternatives • Determine appropriate measure of cost effectiveness • Apply cost model to chosen applications to assess market viability • Identify and examine areas of cost model most sensitive to change • Different regulatory measures • Improvements to technology

37. UAS Applications • Animal tracking • Border Patrol • Local surveillance • Pipeline monitoring • Traffic monitoring • Weather reconnaissance Source: www.cpb.gov

38. Preliminary Cost Model Adapted from a presentation by Ronald Schaefer

39. Preliminary Cost Model • Equipment Manufacture • Basic aircraft (no additional equipment) • Ground control station • Support vehicle • Additional communication hardware/satcom rental • System Operation and Maintenance • Manpower • Training • Salary • Mission Related Expenses • Insurance • Task specific equipment (special sensors) • Fuel • Miscellaneous/Other

40. Problems/Issues • Cost data is hard to get • Many manufacturers are unwilling to disclose specific cost information • Government contract confidentiality • Market confidentiality • Nefarious uses • Not a potential client • Developing detailed cost model • Is anything missing?

41. The Next Step • Add more detail to the cost model • Make more application specific • Obtain more cost data for UAS and their alternatives for each application • Determine effectiveness measures for each application • Determine if cost effectiveness is best measure to use or if there is a better alternative

42. Review Panel Input • Where else do we look for cost data? • Specific people, organizations, publications? • Is our cost model complete? • Any broad categories missing? • Any important sub-categories missing? • Are there alternative measures to cost effectiveness? • Allow to better or more easily assess market viability? • What are good effectiveness measures for our chosen applications?

43. Governance Team Members Nora Darcher Norma Espinosa Scott Fortune Andrea Fuller Managers Keith Florig Leonardo Reyes-Gonzalez

44. Purpose and Goal • Group purpose is to analyze the current system of governance regarding UAS integration and ultimately make recommendations to improve the system considering the principles of good governance • Goal is to advance discussion on whether and how to integrate into the NAS

45. Principles of Good Governance • 8 Characteristics of Good Governance Source: http://www.unescap.org/pdd/prs/ProjectActivities/Ongoing/gg/governance.asp

46. Stakeholders • Identify: • stakeholders as well as their roles and objectives • alliances and competitors • channels and conventions of interaction • Goal is to gain an understanding of how the governance system for the airspace functions

47. Stakeholders

48. Roadblock Analysis • Analyze system to determine roadblocks to UAS integration • Organize roadblocks: • First categorize list: • Technological – Sense and avoid • Organizational – Certification standards • Infrastructural – Air traffic control technology • Public Concern – Privacy infringement and safety

49. Roadblock Analysis (cont’d) • Next prioritize list • Most important to solve • Easiest to solve • Perform historical analysis • Introduction of ultralights • Exempt from many aircraft standards • No license or training required for pilots

50. Expected Results • Make recommendations to address issues on the list by improving “governance system” • Possible recommendations include: • Funding initiatives • Broader stakeholder participation • Database development and information sharing • Increased staffing • Need for outside help or advice • Providing technical assistance • Specific policy initiatives • Creation of NTSB-type agency for UAS crashes