Project on Unmanned Aircraft in the NAS Final Review Panel Meeting

1. Project on Unmanned Aircraft in the NAS Final Review Panel Meeting

2. Integration of Unmanned Aircraft into the National Airspace System A Project Course by Carnegie Mellon University Dept. of Engineering and Public Policy Dept. of Social and Decision Sciences May 1, 2007

3. Expert Review Panel • Tom Curtin, AUVSI • Bret Davis, AUVSI • Lexa Garrett, America West Airlines • Jim Geibel, GAO • David Gerlach, FAA • Tom Henricks, Aviation Week • Ramon Lopez, Aurora Flight Sciences • Edmond Menoche, GAO • Rene Rey, FAA • Melissa Rudinger, Aircraft Owners & Pilots Assn. • James Sizemore, FAA • Larry Thomas, GAO • Dyke Weatherington, DoD/OSD

4. Purpose of CMU Project Courses in Technology and Policy • Analyze a “real world” policy problem involving technology • Combine diverse information and analytic frameworks to derive policy insights • Learning objectives: • Problem decomposition, structuring and formulation • Interdisciplinary problem solving • Communication • Teamwork

5. Examples of past project courses

6. Contributors to our UAS project • 20 undergraduates majoring in: • Engineering • Social Science • Business Administration • 3 Ph.D. student managers • 3 faculty advisors • Expert review panel • Other experts

7. Background for this Project • Increasing demand for UA • Military (many current uses) • Civilian (many potential uses) • Federal Aviation Administration (FAA) is developing a roadmap for integrating UA into the NAS • A few of the issues to be addressed: • Safety and reliability • Public acceptability • Market viability

8. Analysis Areas • Economics • How cost-effective are UA compared to alternative means of providing specific services? • Risk, Technology and Standards • What are the regulatory implications of different approaches to “equivalent level of safety?” • Public Awareness and Perceptions • Are risks of UA of greater public concern than risks of manned aircraft? • Governance • How can the current system for deliberation and decision-making on UA access be improved?

9. Project Outcomes • ~16 person-months of research completed across the four focus areas • Economic model of market viability • Risk model of fatality implications of UA introduction • Better understanding of public awareness & risk perception • Actor & “roadblock” analysis yields insight on deliberative process for UA integration • Regulatory & policy recommendations

10. Economics Team Members: Nathan Diorio-Toth Feng Deng Reiko Baugham Victoria Morton Brad Brown Team Manager: Ryan Kurlinski 10

11. Purpose Assess the market viability of UAS applications using relative cost effectiveness Assess the effect of various regulatory measures on the market viability of UAS applications 11

12. Goals • Develop UAS cost model • Cost components • Airframe • Communications • Insurance • Pilot • Etc. • Apply cost model to chosen applications and alternatives to compare cost • Examine sensitivity of overall cost to changes in each cost component • Estimate cost implications of different regulatory measures and technology improvements 12

13. UAS Applications • Weather Reconnaissance • Alternative: • WC-130J Hercules: high-wing, medium range aircraft • Pipeline monitoring • Alternative: • Concentric sensors: pressure sensitive sensors • Localized Surveillance • Alternative: • Traffic Helicopter: e.g. Bell JetRanger 13

14. Analysis Method • Used triangular distributions to assign probable ranges to each input cost • From this, generated a Probability Density Function • Probability Density Function shows the entire range of possible costs with the associated likelihood of each cost • Allows analysis of the most probable cost advantages 14

15. Importance Analysis Contribution of uncertainty in each input to uncertainty in total cost Triangular probability distributions of all input variables Economic Model

16. Weather Reconnaissance Analyzed the use of Aersonde UAS for Weather Reconnaissance vs. the use of the WC-130J Hercules Aerosonde UAS currently in use for Weather Reconnaissance Hercules WC-130J currently in use by Keesler Air Force Base 16

17. Results: Weather Reconnaissance 12u 10u 8u Probability Density 6u 4u 2u 0 0 100K 200K 300K 400K 500K 600K 700K Alt Cost-UAS Cost Probability Density of UAS Cost Advantage ($/flight hour) 17

18. Results: Weather Reconnaissance 1 0.8 0.6 Importance in Alt Cost-UAS Cost 0.4 0.2 0 Safety Technology Cost Component costs Manpower Cost per gallon Hours per year Com-Link Cost Insurance Rate Gallons per Hour Operational lifetime Alt Cost-UAS Cost Inputs Importance Analysis of Model Inputs Mission Hours per Year Operational Lifetime Com-Link Cost 18

19. Results: Weather Reconnaissance • Key Results: • UAS more cost effective than current manned alternative • Most important inputs in determining overall cost effectiveness: • Mission hours per year • Com link cost • Operational lifetime • Currently available sense-and-avoid equipment cause significant decrease in cost effectiveness, but does not cause the UAS to be more expensive than the manned alternative 19

20. Pipeline Monitoring Analyzed the use of the Aero Environment AeroPuma vs. the use of concentric wire sensors ($6+/m) Note the difference in monitoring style UAS monitors using thermal imaging with each pass and relays pertinent leak info to docking stations Concentric sensors constantly monitor pipeline and relay information

21. Results: Pipeline Monitoring • Key Results: • UAS cheaper depending on number in use • Important to note difference in monitoring styles between UAS and concentric sensor • Important inputs: • Relay/Docking station cost • Number of UASs in use 21

22. Localized Surveillance Application based on the surveillance of a 1km2 area for a short time (~1-3 hours) Considered the use of a Cyber Defense Systems CyberBUG vs. the use of a traffic helicopter For model inputs, considered monitoring a large traffic accident over 2 hours For policy considerations, analyzed the addition of mandated sense-and-avoid hardware to the UAS 22

23. Results: Localized Surveillance PDF of Cost per Mission for UAS Compared with Manned Alternative Probability Density Note: no meaningful overlap 2000 4000 6000 8000 8000 12K 14K 16K 10K Cost per Mission ($) 23

24. Results: Localized Surveillance PDF of Cost per Mission for UAS Compared with Manned Alternative with High-Range Fixed Cost Variance Probability Density Note: still no meaningful overlap 2500 7500 10K 12.5K 20K 22.5K 5000 15K 17.5K Cost per Mission ($) 24

25. Results: Localized Surveillance PDF of Cost per Mission for a Larger UAS Capable of Carrying Sense-and-Avoid Equipment Compared with the Cost of Manned Alternative Probability Density Note: Significant overlap indicating that UAS would likely no longer be a viable alternative to manned craft 0 10K 20K 30K 40K 50K 60K 70K Cost per Mission ($) 25

26. Results: Localized Surveillance Missions per Year Mission Related Costs Flight Hours Per Mission Input Importance for Cost Per Mission Importance of inputs. Input Costs 26

27. Results: Localized Surveillance • Key Results: • UAS less expensive in almost every case • Levelized cost for manned more sensitive than to utilization hours & discount rate than cost for unmanned • UAS cost effectiveness reduced significantly by requirement for sense-and-avoid hardware • Important inputs: • Missions per year • Discount rate • Flight hours per mission 27

28. Policy Implications • Analyzed the effect of the following policies: • Mandated insurance premiums • Mandated use of A/N hardware • (Increased fixed cost) • Mandated record-keeping practices • (Increased yearly cost) • Mandated airframe materials • (Increased fixed cost) • Mandated minimum amount of safety equipment • (Increased fixed cost) • Mandated pilot/operator training 28

29. Policy Implications: Results • All policies except mandated sense-and avoid hardware had little effect on the cost advantage of UAS over manned alternative • Required sense-and-avoid hardware greatly affects cost-effectiveness, however • Localized Surveillance and Pipeline Monitoring would no longer be viable as larger, much more expensive UAS would be necessary 29

30. Risk, Technologies, & Standards Team Members: Samiah Akhtar Jonathan Cornell Nicole Hayward Will Kim Nick Misek Doug Robl Team Manager: Keith Florig 30

31. Purpose • Derive a risk model to explore how risk is related to UAS numbers, dimensions, and flight zones • Research on elements of risk mitigation such as human factors, sense and avoid • Exploration of alternative incident reporting systems Predator Source: http://www.fs.fed.us/psw/news/PSW_News/2005_09/images/uav.gif

32. Technology and Risk Outline • Goals • Risk Modeling • Purpose • Assumptions & Approach • Findings • UAS Risk Mitigation

33. Risk Modeling Purpose • Provide a way of modeling that creates some groundwork for future modeling • Use model to compare relative risk calculations • Pointer to the future, not the answer • Points of interest • Mid-air vs. single-craft crash • Effect of sense and avoid technology • UAS to displace manned aircraft Source: http://www.maximog.com/images/sublevel/uav_left.jpg

34. Risk Modeling Assumptions • Uniform national model • Uniform traffic density • Uniform ground population density • Uniform aircraft per type • Appropriate for: • VFR traffic • Rural, less populated areas • NOT Appropriate for: • Urban settings • Airports • High traffic densities

35. Risk Model

36. Risk Modeling Approach Number of midair collisions: N = total number of aircraft in defined airspace ρ = aircraft traffic density D = diameter of plane (wingspan) S = average aircraft speed P(A) = probability of avoidance (Used for calibration) VFR operations only UAV Picture Source: http://www.evworld.com/press/spider_lion_uav.jpg

37. Risk Modeling: UAs displacing Manned Small risk from unmanned at lower extrema Risk from unmanned at low levels less than decreased risk from manned Single-craft crashes still present less risk than mid-airs

38. Risk Modeling: Mid-Air vs. Single-Craft At some point, manned risk surpasses unmanned risk At low numbers, sense and avoid has little effect Single-Craft generally less risk than mid-air

39. Risk Modeling Conclusions • Mid-air collisions generally have more risk than single-craft crashes • Displacing small to moderate amounts of manned craft represents decrease in risk • Smaller, less reliable UAs can present less risk than larger more reliable manned aircraft • For small numbers of UAs in low traffic densities, sense and avoid has small effect

40. Technology and Risk Outline • Goals • Risk Modeling • UAS Risk Mitigation • Human Factors • Sense and Avoid

41. Human Factors Implications • Risks - Caused Most Number of Accidents • “Sensory Isolation” [McCarley et al] • UAS operator does not receive same sensory cues as manned aircraft operator • Automation • Malfunction of automated components controlled by the UAS operator • Operator Hand-Off • Issues with handing off control of vehicle from one operator or crew to another

42. Human Factors Implications • Recommendations • Training and Procedures • Up to date training as new technology advancements arise • Ensure that operator has accurate knowledge of automated components within UAS • Multimodal displays • Prevent sensory isolation • Allow for audio, visual and speech control • Example: simulated cockpit

43. Detect, Sense and Avoid • Risks • Market impact of single fatal collision • Lack of standardization among DSA systems

44. Detect, Sense and Avoid • Recommendations • Create regulations specific to size, weight, application etc • Testing Periods • Phased Integration

45. Technology and Risk Outline • Issues • Goals • Risk Modeling • UAS Risk Mitigation • Reporting systems

46. Current Reporting Systems • Two Options • NTSB Reporting (as required by FAR) - Accident • NASA ASRS Voluntary Reporting - Incident • Current Implementation • NTSB mandates detailed information when: • Flight control system malfunction, Illness of crewmember, Turbine Engine Failure, In-flight fire, Mid-air collision or Damage in excess of $25,000 to other property • ASRS System is anonymous and does not have any reporting requirements

47. Reporting Recommendations • Initially mandate reporting of all accidents and incidents • Re-evaluate strategy after testing period NTSB - NTSB information helps FAA to assess standards - FAA responds with rules for reporting incidents. - NTSB provides useful information on UAS failures - UAS responds with improved design and engineering Communication Triangle UAS manufacturers FAA

48. Public Awareness & Perceptions Team Members: Darian Ghorbi Jenny Kim Mark Peterson Laura Seitz Patrick Snyder Team Manager: Pete Tengtrakul 48

49. Statement of Purpose • Add the element of public perception to the discussions of UAS in the NAS • Motivation: the fact that there has never been a formal presentation of public perception on the topic • Findings: useful for the creation of regulations and policy implications

50. Objectives • Compare public perceptions of the risks concerning manned and unmanned aircraft • Find demographic groups with certain risk and benefit patterns of UAs • Research implications of opinion of UAs • Create survey to aid in completing objectives